Pith. sign in

REVIEW 4 major objections 75 references

Zero-shot LoRA memory routing works by decoding how each adapter responds to a single frozen base-model prefill.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 21:33 UTC pith:IQWNAQSV

load-bearing objection Solid zero-shot LoRA routing package with a real bench and strong task-skill wins; train-split calibration softens the purest zero-shot claim but does not erase the contribution. the 4 major comments →

arxiv 2607.04118 v1 pith:IQWNAQSV submitted 2026-07-05 cs.LG cs.AI

Parametric Memory Decoding for Zero-Shot Routing in LoRA-Based External Parametric Memory

Fengxian Ji , Zhuohan Xie , Jingpu Yang , Fan Zhang , Zirui Song , Xiuying Chen This is my paper
classification cs.LG cs.AI
keywords zero-shot LoRA routingexternal parametric memoryParametric Memory DecodingPMDRouterresponse energyLoRA bankPEFTPMD-Bench
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether a bank of LoRA modules used as external parametric memory can be selected without a trained router, a retrieval index, or any external embedding space. It organizes PMD-Bench so that document-level, domain-knowledge, and task-skill memories are evaluated under one access protocol, and it reframes zero-shot routing as Parametric Memory Decoding: build a query-conditioned response from frozen-backbone activations and each LoRA’s weights, then decode that response into a score. PMDRouter does this with one adapter-free prefill, scoring each LoRA by scale-normalized linear-response energy. On the benchmark it is the strongest internal-signal zero-shot method in most settings, with very large gains on task-skill routing, while the PMD view shrinks the estimated routing-design search space by about forty times and improves best accuracy relative to direct matching. A reader who wants modular LLM memory without a second routing stack cares because the paper shows that access can live in the interaction of the query activation with the adapter weights themselves.

Core claim

A LoRA-based external parametric memory bank can be addressed in zero-shot fashion when routing is treated as decoding the query-conditioned response each LoRA induces on frozen backbone activations, rather than matching the query to a static adapter descriptor. Instantiating that idea with a parameter-free scale-normalized energy score from a single adapter-free prefill yields the strongest internal-signal top-1 routing accuracy across most multi-granularity settings on PMD-Bench.

What carries the argument

Parametric Memory Decoding (PMD): the routable object is the query-conditioned response ρk(q) = Resp(h(q), θk), scored by a decoder D; PMDRouter sets Resp to the low-rank linear map B A u(x) from one prefill and D to calibrated scale-normalized response energy.

Load-bearing premise

That how strongly each LoRA’s first-order linear update pushes on one frozen prefill activation is a reliable ranking of which memory is right for the query, even when candidate memories cover similar content.

What would settle it

Build a bank of highly overlapping domain or document LoRAs; if scale-normalized response energy systematically mis-ranks the gold adapter while per-LoRA loss probing or oracle selection still recovers the right unit and better downstream answers, the linear-response decoder is not rank-preserving.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • LoRA memory banks can be deployed without maintaining a learned gate, retrieval index, or external representation space for routing.
  • Zero-shot router design collapses from a large coupled matching search into response construction plus decoding, with about a 40× smaller estimated design space and higher best accuracy on PaperQA.
  • Optional training-time signal writing can enlarge response margins so the same inference decoder becomes more reliable.
  • Task-skill LoRAs are highly decodable from parametric responses; knowledge-oriented banks remain competitive with lexical retrieval rather than uniformly replacing it.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Normalized top-1/top-2 response margins could serve as an online confidence signal to fall back to multi-LoRA fusion or text retrieval on ambiguous queries.
  • The same response-decoding view may apply to other PEFT modules whenever a low-rank update can be applied to a frozen backbone activation.
  • For domain banks with high content overlap, signal writing during training may need to be default practice rather than an optional add-on for reliable zero-shot access.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 0 minor

Summary. The paper studies zero-shot routing over LoRA-based external parametric memory without a trained router or external retrieval index. It organizes PMD-Bench (PaperQA, NQ-DomainLoRA, Task-LoRA) across document, domain, and task-skill granularities, and proposes Parametric Memory Decoding (PMD), which reframes routing as constructing and decoding a query-conditioned response object ρ_k(q)=Resp(h(q),θ_k). The main instantiation, PMDRouter, uses a single adapter-free backbone prefill, forms the linear response ρ=BAu(x), and scores adapters by scale-normalized, train-split-calibrated response energy (Eqs. 5–7). On three backbones, PMDRouter is the strongest internal-signal zero-shot router in most settings (especially Task-LoRA), with supporting path ablations, margin-quintile analyses, optional signal-writing training, and a design-space complexity comparison versus direct matching.

Significance. If the results hold under a cleanly stated zero-shot protocol, the work is a useful contribution to modular LLM memory: it unifies several prior zero-shot LoRA routers as restricted PMD paths (Appendix B.2), provides a multi-granularity EPM access benchmark, and shows that a single-prefill internal signal can be competitive without a learned gate. Strengths include a broad baseline suite (backbone-only, LoRA-only, joint, and text retrieval), margin-based separability analysis (Fig. 4), path ablations (Figs. 3, 5), and an anonymous code release. The practical significance is tempered by strong BM25/embedding baselines on knowledge tasks and by the fact that the headline “parameter-free zero-shot” claim depends on train-split calibration, which needs clearer isolation before the result can be treated as a clean protocol win.

major comments (4)
  1. Section 4.3 and Eqs. (6)–(7) present PMDRouter as a parameter-free zero-shot score from one adapter-free prefill, but then apply Calib_k estimated on the training split and fixed at evaluation; Appendix tables further compare under a “matched train-split calibration setting.” This is not a learned router, yet it is a memory-unit-specific correction fit on the same partition that produced each LoRA. The central “strongest internal-signal zero-shot” claim (Abstract; §5.2; Table 1) therefore needs an explicit uncalibrated vs calibrated ablation of raw ∥BAu∥²/(∥u∥²∥BA∥_F²+ε) on all three benchmarks and backbones. Without that, it is unclear how much of the reported top-1 margins—especially the mixed NQ-DomainLoRA results—depend on Calib_k rather than the response object itself.
  2. Table 1 and §5.2 show that BM25 (and sometimes embedding retrieval) often match or beat all parametric zero-shot routers on PaperQA and especially NQ-DomainLoRA, while PMDRouter’s clearest gains are on Task-LoRA. The paper acknowledges this, but the abstract and conclusion still frame the work as demonstrating feasibility of zero-shot LoRA routing for EPM access in general. Please restate the main claim more precisely as “strongest among internal-signal zero-shot routers under the stated protocol,” and add a short discussion of when parametric response decoding is preferable to lexical retrieval (e.g., skill banks vs long-document knowledge), so the contribution is not over-read as replacing retrieval for knowledge EPM.
  3. Table 3’s ~40× complexity reduction is partly definitional: under the PMD view the query and memory axes are collapsed into a response object by construction (§4.2, Eq. 2), so the reduced search space (~3 vs ~56–4.2×10^5) follows from the chosen ontology as much as from empirical simplification. The accompanying accuracy gain (0.636→0.669 on PaperQA) is useful, but the complexity claim should be rephrased as a design-space reorganization with an explicit counting protocol (what counts as a free choice, how coupling is measured), or moved to discussion rather than listed as a primary contribution on equal footing with routing accuracy.
  4. §5.4 / Table 2 vs surrounding text are inconsistent on the best signal-writing variant: the prose claims “QMean-All-LogC” best at 69.78 Acc / 0.2637 NormGap, while Table 2 reports QMean-Raw 0.689 Acc / 0.554 NormGap as best and QMean-Log 0.676 / 0.527, with NormGap scales that do not match the prose numbers. Naming also drifts (QMean-Log vs QMean-All-LogC; Base 0.613 vs main PaperQA 0.613 for Qwen3-4B). Please reconcile table and text, define each variant once, and report the same metrics under the same inference router used in Table 1 so the training-side claim is verifiable.

Circularity Check

1 steps flagged

Main routing results are empirical; only the ~40× design-space reduction is partly definitional under the PMD ontology.

specific steps
  1. self definitional [Section 5.5 / Table 3]
    "Under the PMD view, the query and memory axes collapse into a response-centered formulation: the main remaining choices are reduced to the response instantiation and the decoder. This reduces the estimated search space to ∼3, coupling degree to ∼1, and normalized complexity to ∼0.025. Despite this smaller design space, PMD also supports stronger routing performance on PaperQA, improving the best accuracy from 0.636 under the direct matching view to 0.669."

    PMD is defined (Eq. 2–4) as routing via Resp(h(q), θk) rather than separate query/memory matching. Counting design axes after that collapse necessarily yields a much smaller search space; the ~40× complexity reduction is therefore largely by the chosen ontology, not an independent empirical measurement of search cost. (The accompanying Acc. gain is empirical and not circular.)

full rationale

The paper’s load-bearing claims are top-1 routing accuracy of PMDRouter versus external zero-shot and retrieval baselines on held-out PMD-Bench splits (Table 1, Sections 5.1–5.2). Those scores are measured against gold memory labels and do not reduce by construction to fitted targets or to a self-citation uniqueness theorem. Calibration from the train split (Section 4.3) softens a pure zero-shot reading but is a fixed affine correction used at evaluation, not a quantity the paper pretends to predict. Self-citations in the bibliography are not load-bearing for the method. The only mild circularity is the complexity narrative (Table 3 / §5.5): PMD is defined as collapsing query- and memory-side axes into a response object, so the reported drop from ~1.00 to ~0.025 normalized complexity largely follows from that re-counting rather than an independent measurement. Best Acc. 0.636→0.669 remains an empirical side claim. Overall this is a normal non-circular ML methods paper with one secondary definitional complexity count.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 3 invented entities

The central claim rests on standard LoRA linear algebra plus the modeling choice that query-conditioned response energy is a good routing surrogate, plus several implementation knobs (pooling, layers, train-split calibration, optional write loss). No new physical entities; invented constructs are methodological (response object, bench, router).

free parameters (4)
  • calibration statistics Calib_k (train-split)
    Per-LoRA calibration estimated from training split and fixed at eval; affects absolute scores and ranking stability.
  • epsilon in energy denominator
    Numerical stabilizer in scale-normalized energy (Eq. 6); small but part of the decoder.
  • signal-writing weight lambda
    Balances task loss vs write loss when training-side shaping is used (Eq. 8).
  • pooling / layer / module set choices
    SemPool rule, adapted layer set S_k, and module aggregation are design choices that change the response object.
axioms (4)
  • domain assumption LoRA update is low-rank linear: Delta W = B A, and first-order response B A u(x) is an adequate routing surrogate for full adapter effect.
    Section 4.3; higher-order and generation-path effects are omitted as rank-preserving constants.
  • ad hoc to paper Zero-shot routing excludes routing-specific training data, retrieval indexes, and external representation spaces.
    Definitional protocol in Sections 1 and 3.1; train-split calibration still used for score centering.
  • domain assumption Target memory is uniquely addressable by max response score among K candidates.
    Hard top-1 routing setup throughout experiments; soft routing mentioned but not primary.
  • standard math Standard transformer PEFT / LoRA training and causal LM loss.
    Background used for bank construction and optional L_task.
invented entities (3)
  • Parametric Memory Decoding (PMD) framework / response object rho_k(q) no independent evidence
    purpose: Unify zero-shot LoRA routing as response construction plus decoding rather than static matching.
    Methodological construct; independent evidence is empirical routing accuracy, not an external physical prediction.
  • PMD-Bench (PaperQA, NQ-DomainLoRA, Task-LoRA) no independent evidence
    purpose: Provide multi-granularity EPM access evaluation.
    New benchmark suite constructed by the authors from public sources; value is evaluative, not a natural entity.
  • PMDRouter (scale-normalized response energy decoder) no independent evidence
    purpose: Concrete parameter-free router from one backbone prefill.
    Instantiation of PMD; validated only within the paper’s experiments.

pith-pipeline@v1.1.0-grok45 · 28828 in / 3281 out tokens · 32800 ms · 2026-07-11T21:33:40.264124+00:00 · methodology

0 comments
read the original abstract

With the rise of parametric memory, LoRA-based External Parametric Memory (EPM) has emerged as a modular solution, but existing routing methods often introduce additional training, deployment, and maintenance overhead. This raises a natural question: can a LoRA-based EPM bank be routed without maintaining an additional routing component? However, existing zero-shot LoRA routing methods still face two problems under the EPM setting: (1) their evaluations are scattered across different task settings rather than organized around EPM access, and (2) their routing signals lack a unified perspective to guide systematic improvement. To address these problems, we organize PMD-Bench, covering document-level, domain-level knowledge, and task-skill, and propose Parametric Memory Decoding (PMD), the first framework designed to systematically improve zero-shot LoRA routing by reframing it as decoding activations over external parametric memory. Based on PMD, we further instantiate PMDRouter, which scores each LoRA by its response magnitude from a single base-model prefill. Experiments on PMD-Bench show that PMDRouter achieves the strongest internal-signal performance across multiple zero-shot routing settings. These results demonstrate the feasibility of zero-shot LoRA routing and suggest that PMD can serve as a general framework for improving zero-shot routing methods. Sources: Github (https://anonymous.4open.science/r/Parametric-Memory-Decoding-872A/)

Figures

Figures reproduced from arXiv: 2607.04118 by Fan Zhang, Fengxian Ji, Jingpu Yang, Xiuying Chen, Zhuohan Xie, Zirui Song.

Figure 1
Figure 1. Figure 1: Three LoRA routing paradigms: (a) learned routing networks, (b) retrieval-augmented routing, and (c) zero-shot LoRA routing, where scores are computed from the query-side signal h(q) and observable LoRA weights (Ak, Bk). PMD defines the routable response object as ρk(q) = Resp(h(q),(Ak, Bk)), and decodes it as sk = D(ρk(q)). and observable LoRA-side information. Representative methods such as Arrow, SEQR, … view at source ↗
Figure 2
Figure 2. Figure 2: Overview of PMD and PMDRouter. (A) PMD Framework: Response Object and Decoding; [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Routing accuracy across signal paths. Moving from single-side cues to joint and PMD [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: PMD response margin predicts routing success. Queries are grouped by normalized [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Ablation Study. Accuracy of different decoder adaptation settings across backbone mod￾els: Backbone only, LoRA only, Projection only, and Full decoder. To clarify PMDRouter’s gains, we compare four inference-time decoder settings on Pa￾perQA: Backbone-only, LoRA-only, Projection￾only, and the full decoder. As shown in [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 28 linked inside Pith

  1. [1]

    Araujo, M

    V . Araujo, M. F. Moens, and T. Tuytelaars. Learning to route for dynamic adapter composition in continual learning with language models. InFindings of the Association for Computational Linguistics: EMNLP 2024, pages 687–696, 2024

    2024

  2. [2]

    S. Back, D. Lee, N. Kang, T. Lee, S. K. Hong, Y . Gwon, and S. Ahn. Understanding lora as knowledge memory: An empirical analysis, 2026

    2026

  3. [3]

    Badawi, M

    M. Badawi, M. Abushanab, S. Bhat, and A. Maier. Review of zero-shot and few-shot ai algorithms in the medical domain.arXiv preprint arXiv:2406.16143, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    H. Bai, H. Wang, S. Chen, Z. Chen, L.-A. Tang, W. Cheng, Y . Fu, and H. Chen. Learning to route: A rule-driven agent framework for hybrid-source retrieval-augmented generation. InProceedings of the ACM Web Conference 2026, pages 4338–4349, 2026

    2026

  5. [5]

    M. Bini, O. Bohdal, U. Michieli, Z. Akata, M. Ozay, and T. Ceritli. Memlora: Distilling expert adapters for on-device memory systems.arXiv preprint arXiv:2512.04763, 2025

    arXiv 2025

  6. [6]

    E. L. Buehler and M. J. Buehler. X-lora: Mixture of low-rank adapter experts, a flexible framework for large language models with applications in protein mechanics and molecular design.APL Machine Learning, 2(2), 2024

    2024

  7. [7]

    S. Cai, Y . Shu, and W. Wang. Dynamic routing networks. Inproceedings of the IEEE/CVF winter conference on applications of computer vision, pages 3588–3597, 2021

    2021

  8. [8]

    J. Cao, Z. Fan, Z. Wang, T. Lin, Z. Zhao, R. Yan, W. Zhang, F. Shao, H. Wang, J. Xiao, et al. Comol: Efficient mixture of lora experts via dynamic core space merging.arXiv preprint arXiv:2603.00573, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    J. Cao, Y . Ma, X. Li, Q. Ren, and X. Chen. Task-specific efficiency analysis: When small language models outperform large language models, 2026

    2026

  10. [10]

    J. Cao, J. Wang, R. Wei, Q. Guo, K. Chen, B. Zhou, and Z. Lin. Memory decoder: A pretrained, plug-and-play memory for large language models.arXiv preprint arXiv:2508.09874, 2025

    arXiv 2025

  11. [11]

    M.-Y . Chen, T. T. U. Hoang, M. Hahn, and M. S. Sarfraz. When does LoRA reuse work? theoretical limits and mechanisms for recycling LoRAs without data access.Transactions on Machine Learning Research, 2026

    2026

  12. [12]

    S. Chen, Z. Jie, and L. Ma. Llava-mole: Sparse mixture of lora experts for mitigating data conflicts in instruction finetuning mllms.arXiv preprint arXiv:2401.16160, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    M. Cui, J. Yang, F. Ji, Q. Jiang, Z. Shi, J. Wang, Z. Song, F. Koto, and X. Chen. Textalign: Preference alignment for text rendering with hierarchical rewards.arXiv preprint arXiv:2605.19320, 2026

    Pith/arXiv arXiv 2026

  14. [14]

    Dhasade, A.-M

    A. Dhasade, A.-M. Kermarrec, I. Pavlovic, D. Petrescu, R. Pires, M. Randl, and M. de V os. Effective lora adapter routing using task representations.arXiv preprint arXiv:2601.21795, 2026

    arXiv 2026

  15. [15]

    Fedus, B

    W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity.Journal of Machine Learning Research, 23(120):1–39, 2022

    2022

  16. [16]

    Fleshman and B

    W. Fleshman and B. Van Durme. Lora-augmented generation (lag) for knowledge-intensive language tasks. arXiv preprint arXiv:2507.05346, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Fleshman and B

    W. Fleshman and B. Van Durme. Seqr: Secure and efficient qr-based lora routing.arXiv preprint arXiv:2509.18093, 2025. 10

    arXiv 2025

  18. [18]

    Fleshman and B

    W. Fleshman and B. Van Durme. Spectr: Dynamically composing lm experts with spectral routing.arXiv preprint arXiv:2504.03454, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    C. Gao, K. Chen, J. Rao, R. Liu, B. Sun, Y . Zhang, D. Peng, X. Guo, and V . Subrahmanian. Mola: Moe lora with layer-wise expert allocation. InFindings of the Association for Computational Linguistics: NAACL 2025, pages 5097–5112, 2025

    2025

  20. [20]

    Z. Han, H. Wang, Z. Zhang, X. Dai, X. Liu, and J. Lui. Hilora: Adaptive hierarchical lora routing for training-free domain generalization.arXiv preprint arXiv:2510.12266, 2025

    arXiv 2025

  21. [21]

    Q. He, Y . Wang, X. Wang, W. Xu, F. Li, K. Yang, and L. Ma. Routing optimization with deep reinforcement learning in knowledge defined networking.IEEE Transactions on Mobile Computing, 23(2):1444–1455, 2023

    2023

  22. [22]

    Heinzerling and K

    B. Heinzerling and K. Inui. Language models as knowledge bases: On entity representations, storage capacity, and paraphrased queries. InProceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 1772–1791, 2021

    2021

  23. [23]

    Huang, Z

    Y . Huang, Z. Yang, Z. Wang, J. Qi, R. Yu, X. Fan, and C. Wang. Hybrid routing for a mixture of lora experts. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31211–31219, 2026

    2026

  24. [24]

    Jeong, W

    W. Jeong, W. Lee, and K.-J. Yoon. Preference-aligned lora merging: Preserving subspace coverage and addressing directional anisotropy.arXiv preprint arXiv:2603.26299, 2026

    arXiv 2026

  25. [25]

    F. Ji, J. Yang, Z. Song, L. Gao, J. Liang, Z. Chen, J. Zhang, and X. Chen. Servimage: An image generation and editing benchmark from real-world commercial imaging services.ACL26 Main, 2026

    2026

  26. [26]

    F. Ji, J. Yang, Z. Song, Y . Wang, Z. Cui, Y . Li, Q. Jiang, and X. Chen. Finestate-bench: Benchmarking state-conditioned grounding for fine-grained gui state setting.ACL26 Findings, 2026

    2026

  27. [27]

    P. Jin, P. Shu, S. Song, S. Kim, Q. Xiao, C. Chen, T. Liu, X. Li, and Q. Li. Beyond adapter retrieval: Latent geometry-preserving composition via sparse task projection. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 22417–22425, 2026

    2026

  28. [28]

    Kappiyath, A

    A. Kappiyath, A. Chaudhuri, J. Li, A. K. JAISWAL, X. Wang, L. Shen, S. Kundu, X. Zhu, and L. Yin. Maple: Masked adapter prototype learning for ood generalization

  29. [29]

    R. Kong, Q. Li, X. Fang, Q. Feng, Q. He, Y . Dong, W. Wang, Y . Li, L. Kong, and Y . Liu. Lora- switch: Boosting the efficiency of dynamic llm adapters via system-algorithm co-design.arXiv preprint arXiv:2405.17741, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    L. M. Lazier, A. Dhar, V . Stambolic, and L. Cavigelli. Ac-lora:(almost) training-free access control-aware multi-modal llms.arXiv preprint arXiv:2505.11557, 2025

    arXiv 2025

  31. [31]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. ocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020

    2020

  32. [32]

    D. Li, Y . Ma, N. Wang, Z. Ye, Z. Cheng, Y . Tang, Y . Zhang, L. Duan, J. Zuo, C. Yang, et al. Mixlora: Enhanc- ing large language models fine-tuning with lora-based mixture of experts.arXiv preprint arXiv:2404.15159, 2024

    Pith/arXiv arXiv 2024

  33. [33]

    D. Li, N. Wang, Z. Zhang, H. Yin, L. Duan, M. Xiao, and M. Tang. Dynmole: Boosting mixture of lora experts fine-tuning with a hybrid routing mechanism.arXiv preprint arXiv:2504.00661, 2025

    Pith/arXiv arXiv 2025

  34. [34]

    G. Li, Z. Xi, Z. Zhang, B. Hong, T. Gui, Q. Zhang, and X.-J. Huang. Loracoe: Improving large language model via composition-based lora expert. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31278–31292, 2025

    2025

  35. [35]

    Q. Li, R. Kong, Y . Li, H. Cai, S. Wang, L. Kong, G. Chen, and D. Yin. Adafuse: Accelerating dynamic adapter inference via token-level pre-gating and fused kernel optimization. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31680–31688, 2026

    2026

  36. [36]

    X. Li, Y . Ma, Y . Huang, X. Wang, Y . Lin, and C. Zhang. Synergized data efficiency and compression (sec) optimization for large language models. In2024 4th International Conference on Electronic Information Engineering and Computer Science (EIECS), pages 586–591, 2024. 11

    2024

  37. [37]

    X. Li, Y . Ma, K. Ye, J. Cao, M. Zhou, and Y . Zhou. Hy-Facial: hybrid feature extraction by dimensionality reduction methods for enhanced facial expression classification. In W. Osten and E. Mamut, editors, Eighteenth International Conference on Machine Vision (ICMV 2025), volume 14114, page 141140R. International Society for Optics and Photonics, SPIE, 2026

    2025

  38. [38]

    Z. Li, Z. Duan, D. Chen, C. Chen, D. Chen, Y . Li, and Y . Chen. Autolora: Automatic lora retrieval and fine-grained gated fusion for text-to-image generation.arXiv preprint arXiv:2508.02107, 2025

    Pith/arXiv arXiv 2025

  39. [39]

    H. Lu, C. Zhao, J. Xue, L. Yao, K. Moore, and D. Gong. Little by little: Continual learning via incremental mixture of rank-1 associative memory experts.arXiv preprint arXiv:2506.21035, 2025

    Pith/arXiv arXiv 2025

  40. [40]

    Mallen, A

    A. Mallen, A. Asai, V . Zhong, R. Das, D. Khashabi, and H. Hajishirzi. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. InProceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pages 9802–9822, 2023

    2023

  41. [41]

    Muqeeth, H

    M. Muqeeth, H. Liu, Y . Liu, and C. Raffel. Learning to route among specialized experts for zero-shot generalization.arXiv preprint arXiv:2402.05859, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Ostapenko, Z

    O. Ostapenko, Z. Su, E. M. Ponti, L. Charlin, N. L. Roux, M. Pereira, L. Caccia, and A. Sordoni. Towards modular llms by building and reusing a library of loras.arXiv preprint arXiv:2405.11157, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Page-Caccia, E

    L. Page-Caccia, E. M. Ponti, Z. Su, M. Pereira, N. Le Roux, and A. Sordoni. Multi-head adapter routing for cross-task generalization.Advances in Neural Information Processing Systems, 36:56916–56931, 2023

    2023

  44. [44]

    E. M. Ponti, A. Sordoni, Y . Bengio, and S. Reddy. Combining parameter-efficient modules for task-level generalisation. InProceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 687–702, 2023

    2023

  45. [45]

    Prabhakar, Y

    A. Prabhakar, Y . Li, K. Narasimhan, S. Kakade, E. Malach, and S. Jelassi. Lora soups: Merging loras for practical skill composition tasks. InProceedings of the 31st International Conference on Computational Linguistics: Industry Track, pages 644–655, 2025

    2025

  46. [46]

    J. Qiao, W. Meng, Y . Cheng, Z. Lin, Z. Zhang, X. Tan, J. Gong, K. Shao, and Y . Xie. Memory intelligence agent.arXiv preprint arXiv:2604.04503, 2026

    Pith/arXiv arXiv 2026

  47. [47]

    Sawada, D

    K. Sawada, D. Kotani, and Y . Okabe. Network routing optimization based on machine learning using graph networks robust against topology change. In2020 International conference on information networking (ICOIN), pages 608–615. IEEE, 2020

    2020

  48. [48]

    Shazeer, A

    N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. Le, G. Hinton, and J. Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538, 2017

    Pith/arXiv arXiv 2017

  49. [49]

    Shukla, A

    S. Shukla, A. Sriram, M. K. Narayanaswamy, and H. Jain. qa-flora: Data-free query-adaptive fusion of loras for llms. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 32992–33000, 2026

    2026

  50. [50]

    Sprechmann, S

    P. Sprechmann, S. M. Jayakumar, J. W. Rae, A. Pritzel, A. P. Badia, B. Uria, O. Vinyals, D. Hassabis, R. Pascanu, and C. Blundell. Memory-based parameter adaptation.arXiv preprint arXiv:1802.10542, 2018

    Pith/arXiv arXiv 2018

  51. [51]

    Z. Su, F. Mo, G. Liang, J. Zhang, B. Wen, P. Tiwari, and J.-Y . Nie. Tensorized clustered lora merging for multi-task interference.arXiv preprint arXiv:2508.03999, 2025

    Pith/arXiv arXiv 2025

  52. [52]

    Y . Tian, B. Zhang, Z. Tu, and D. Chu. Adapters selector: Cross-domains and multi-tasks lora modules inte- gration usage method. InProceedings of the 31st International Conference on Computational Linguistics, pages 593–605, 2025

    2025

  53. [53]

    Valadarsky, M

    A. Valadarsky, M. Schapira, D. Shahaf, and A. Tamar. Learning to route. InProceedings of the 16th ACM workshop on hot topics in networks, pages 185–191, 2017

    2017

  54. [54]

    Wadhwa, R

    H. Wadhwa, R. Seetharaman, S. Aggarwal, R. Ghosh, S. Basu, S. Srinivasan, W. Zhao, S. Chaudhari, and E. Aghazadeh. From rags to rich parameters: Probing how language models utilize external knowledge over parametric information for factual queries.arXiv preprint arXiv:2406.12824, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    H. Wang, Y . Li, S. Wang, G. Chen, and Y . Chen. Milora: Harnessing minor singular components for parameter-efficient llm finetuning. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 4823–4836, 2025. 12

    2025

  56. [56]

    P. Wang, Z. Li, N. Zhang, Z. Xu, Y . Yao, Y . Jiang, P. Xie, F. Huang, and H. Chen. Wise: Rethinking the knowledge memory for lifelong model editing of large language models.Advances in Neural Information Processing Systems, 37:53764–53797, 2024

    2024

  57. [57]

    T. Wang, A. Roberts, D. Hesslow, T. Le Scao, H. W. Chung, I. Beltagy, J. Launay, and C. Raffel. What language model architecture and pretraining objective works best for zero-shot generalization? In International Conference on Machine Learning, pages 22964–22984. PMLR, 2022

    2022

  58. [58]

    R. Wei, J. Cao, J. Wang, J. Kai, Q. Guo, B. Zhou, and Z. Lin. Mlp memory: A retriever-pretrained memory for large language models.arXiv preprint arXiv:2508.01832, 2025

    arXiv 2025

  59. [59]

    X. Wu, S. Huang, and F. Wei. Mixture of lora experts.arXiv preprint arXiv:2404.13628, 2024

    Pith/arXiv arXiv 2024

  60. [60]

    F. Xia, M. Liao, Y . Fang, D. Li, Y . Xie, W. Li, Y . Li, D. Xia, and J. Huang. Cross-lora: A data-free lora transfer framework across heterogeneous llms.arXiv preprint arXiv:2508.05232, 2025

    Pith/arXiv arXiv 2025

  61. [61]

    J. Yang, F. Ji, Z. Lai, Z. Cui, G. Ouyang, Q. Jiang, F. Zhang, M. Peng, Q. Xie, P. Nakov, et al. Labguard: Grounding natural-language laboratory rules into runtime guards for embodied laboratory agents.arXiv preprint arXiv:2606.31045, 2026

    Pith/arXiv arXiv 2026

  62. [62]

    J. Yang, F. Ji, Z. Lai, J. Wu, M. Cui, and Y . Wang. Zero-parameter geometric gating for temporally stable low-altitude uav video semantic segmentation.arXiv preprint arXiv:2606.09162, 2026

    Pith/arXiv arXiv 2026

  63. [63]

    J.-S. Yang, Z. Zeng, and Z. Shen. Neural-symbolic dual-indexing architectures for scalable retrieval- augmented generation.IEEE Access, 13:210507–210519, 2025

    2025

  64. [64]

    Zhang, Z

    F. Zhang, Z. Li, S. Peng, and Y . Chen. When alpha disappears: A one-switch benchmark for decision-time leakage in financial backtests.arXiv preprint arXiv:2605.23959, 2026

    Pith/arXiv arXiv 2026

  65. [65]

    Zhang, J

    F. Zhang, J. Luo, Z. Zhang, S. Huang, Z. Liu, and Y . Chen. Beyond visual realism: Toward reliable financial time series generation.arXiv preprint arXiv:2601.12990, 2026

    arXiv 2026

  66. [66]

    Zhang, M

    F. Zhang, M. Song, R. Elbadry, Y . Chen, S. Wang, Y . Zhou, X. Zheng, Y . He, Y . Dai, G. Georgiev, et al. Finreporting: An agentic workflow for localized reporting of cross-jurisdiction financial disclosures.arXiv preprint arXiv:2604.05966, 2026

    Pith/arXiv arXiv 2026

  67. [67]

    Zhang, Y

    H. Zhang, Y . Zhang, X. Li, W. Shi, H. Xu, H. Liu, Y . Wang, L. Shang, Q. Liu, Y . Liu, et al. Evaluating the external and parametric knowledge fusion of large language models.arXiv preprint arXiv:2405.19010, 2024

    Pith/arXiv arXiv 2024

  68. [68]

    Zhang and J

    H. Zhang and J. Zhou. Unraveling lora interference: Orthogonal subspaces for robust model merging. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26459–26472, 2025

    2025

  69. [69]

    Zhang, K

    X. Zhang, K. Yang, C. Li, H. Li, Q. Wei, J. Tsujii, and S. Ananiadou. Memadapter: Fast alignment across agent memory paradigms via generative subgraph retrieval.arXiv preprint arXiv:2602.08369, 2026

    arXiv 2026

  70. [70]

    Zhang and R

    Y . Zhang and R. Li. Dlp-lora: Efficient task-specific lora fusion with a dynamic, lightweight plugin for large language models.arXiv preprint arXiv:2410.01497, 2024

    Pith/arXiv arXiv 2024

  71. [71]

    Zhang, Q

    Z. Zhang, Q. Dai, X. Bo, C. Ma, R. Li, X. Chen, J. Zhu, Z. Dong, and J.-R. Wen. A survey on the memory mechanism of large language model-based agents.ACM Transactions on Information Systems, 43(6):1–47, 2025

    2025

  72. [72]

    Z. Zhao, L. Gan, G. Wang, Y . Hu, T. Shen, H. Yang, K. Kuang, and F. Wu. Retrieval-augmented mixture of lora experts for uploadable machine learning.arXiv preprint arXiv:2406.16989, 2024

    Pith/arXiv arXiv 2024

  73. [73]

    Z. Zhao, L. Gan, G. Wang, W. Zhou, H. Yang, K. Kuang, and F. Wu. Loraretriever: Input-aware lora retrieval and composition for mixed tasks in the wild. InFindings of the Association for Computational Linguistics: ACL 2024, pages 4447–4462, 2024

    2024

  74. [74]

    Y . Zhou, Z. Zhao, H. Li, S. Du, J. Yao, Y . Zhang, and Y . Wang. Exploring training on heterogeneous data with mixture of low-rank adapters.arXiv preprint arXiv:2406.09679, 2024

    Pith/arXiv arXiv 2024

  75. [75]

    Zhuang, Y

    Y . Zhuang, Y . Shen, Y . Bian, Q. Su, S. Ji, Y . Shi, and F. Miao. Ld-mole: Learnable dynamic routing for mixture of lora experts.arXiv preprint arXiv:2509.25684, 2025. 13 A Dataset We construct three benchmark datasets: NQ-DomainLoRA, PaperQA, and Task-LoRA. All training data are formatted into a unified two-turn messages schema for LoRA training, while...

    arXiv 2025