On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View

Dacheng Tao; Fushuo Huo; Jie Gui; Junxiao Wang; Laizhong Cui; Song Guo; Tao Guo

REVIEW 3 major objections 1 minor 40 references

On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View

T0 review · 3 major / 1 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The full text delivered with this submission is not the federated-LLM survey announced in the abstract; it is an experimental study that proposes a unified interface and taxonomy for attribute-filtered approximate nearest-neighbor search.

desk verdict The submission is not the paper it claims to be: the abstract describes a FedLLM survey, but the full text is a different paper on filtered ANN search, so there is nothing to review. read the letter →

arxiv 2508.16261 v1 pith:UQZKIKSN submitted 2025-08-22 cs.LG

Tao Guo , Junxiao Wang , Fushuo Huo , Laizhong Cui , Song Guo , Jie Gui , Dacheng Tao This is my paper · ORCID

classification cs.LG

keywords federatedlearninglargelanguagemodelsapproximatenearestneighborsearchattributefilteringvectordatabasebenchmarktaxonomy

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The submitted document consists of two mismatched parts. The abstract announces a taxonomy of federated post-training methods for large language models, grouped by whether they use white-box, gray-box, or black-box access to the model. The full text is a different paper: it builds a unified interface for attribute-filtered approximate nearest-neighbor search, classifies algorithms by attribute type and filtering strategy, and benchmarks 10 algorithms and 12 methods on four datasets holding up to 10 million items. The contribution that can actually be checked is therefore the filtering-ANN study, not the FedLLM survey. That study matters because vector search with structured attribute constraints underpins retrieval-augmented generation, recommendation, and vector databases.

What carries the argument

The central objects are the unified Filtering ANN search interface and the two-axis taxonomy. The interface fixes the input/output semantics so different algorithms can be compared head-to-head; the taxonomy organizes the field by attribute types (for example, range predicates versus categorical predicates) and by when filtering happens relative to the search. A component-level analysis of index structures, pruning strategies, and entry-point selection then explains why methods differ in speed and recall on the same workload.

What would settle it

Compare the full text's title and content with the abstract: the mismatch is already visible without any experiment. For the benchmark itself, a concrete falsifier would be adding a filtering-ANN method whose filtering strategy falls outside the proposed taxonomy and showing that it wins on the same four datasets, or showing that the relative ranking of the ten algorithms changes when a different 10-million-item dataset is used.

Watch

Extended reading notes

Core claim

The full-text authors claim that the scattered filtering-ANN methods of the last few years can be brought under one interface and one taxonomy. Their taxonomy is built on two axes—the attribute types being filtered and the filtering strategy used—and they identify index structure, pruning strategy, and entry-point selection as the components that separate methods and explain their tradeoffs. On four datasets with up to 10 million items and selectivity levels from 0.1% to 100%, the authors measure how each component affects efficiency and quality and condense the results into practical method-selection guidelines. None of this appears in the abstract, which describes an unrelated survey.

Load-bearing premise

The announced survey has no body to stand on, and the benchmark actually present assumes that its ten algorithms and four datasets span the filtering-ANN design space well enough for the taxonomy and guidelines to generalize.

Editorial extensions

If this is right

Any new filtering-ANN index can be positioned against ten existing algorithms and twelve method variants under a common interface, making method comparisons replicable.
Because the taxonomy separates attribute type from filtering strategy, system builders can match a workload's predicate shape to the algorithm family most likely to handle it.
The component analysis points to pruning and entry-point selection as the main levers of cost; tuning those components is thus the most direct route to speedups.
The practical guidelines give vector-database and RAG developers a starting choice set for selectivity regimes between 0.1% and 100%, rather than relying on unbenchmarked defaults.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the intended paper is the FedLLM survey named in the abstract, no assessment of that survey's claims is possible from this submission; the abstract and full text would need to be reconciled before its white-box/gray-box/black-box taxonomy can be taken seriously.
The filtering-ANN benchmark's practical conclusions are likely to transfer to workloads with similar attribute distributions; testing the same ten algorithms on datasets with mixed predicate types or very low selectivity would be a direct extension.
The same unified-interface approach could be extended to learned or hybrid indexes, but the paper gives no evidence that its taxonomy already includes them; a reader should not assume coverage beyond what is listed.
A mismatch of this kind, if not resolved, means the public record will carry two different claims under one entry; resolving the discrepancy is a prerequisite for any downstream reader who wants to cite either the survey or the benchmark.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

First, the punchline: the submission is not the paper it claims to be. The abstract promises a comprehensive survey of federated post-tuning of LLMs, with a two-axis taxonomy and white/gray/black-box classes; the full text is a different paper, "Attribute Filtering in Approximate Nearest Neighbor Search," by a different author team, carrying a different arXiv ID. The body contains no section defining the FedLLM taxonomy, no representative method families, no search methodology, and no completeness analysis. The central claim has no supporting text.

I can't review the survey described in the abstract because it doesn't exist in this artifact. The abstract's framing—model-access and parameter-efficiency axes—is a reasonable way to organize the FedLLM literature, and a good survey on those lines would be genuinely useful. But an abstract alone can't support judgments about coverage, accuracy, or novelty. The reader's UNVERDICTED is the correct verdict.

Credit where it's due: the body text, read on its own, is a real experimental study of filtered ANN search. It proposes a unified interface and taxonomy, benchmarks 10 algorithms across 4 datasets, and ships code in a public repository. That is reproducible work. But it belongs to a different submission under its own title; as evidence for the FedLLM survey, it's irrelevant.

The mismatch is not a subtle argumentative flaw. It's a desk-reject-level internal inconsistency: the abstract and body are different papers. This is a missing-support condition, not a disagreement with the authors' conclusions. No referee can evaluate the claimed taxonomy from this material. I would not send this to peer review. The authors should resubmit with the correct full text (or submit the ANN paper properly). If the FedLLM survey actually exists and is as comprehensive as advertised, it would deserve a serious referee—but that is a conditional, not a property of this submission.

Bottom line: desk reject with an invitation to resubmit a corrected version. The abstract's taxonomy is worth reading if it ever materializes; this artifact is not.

Referee Report

3 major / 1 minor

Summary. The submission claims to be a comprehensive survey of federated post-training of large language models (FedLLM), proposing a taxonomy based on model access (white-box, gray-box, black-box) and parameter-efficiency optimization. However, the supplied full text is a different paper: 'Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study' by Mocheng Li, Xiao Yan, Baotong Lu, Yue Zhang, James Cheng, and Chenhao Ma (arXiv:2508.16263v2, cs.DB). The body contains no FedLLM taxonomy, no representative FedLLM methods, no survey tables, and no discussion of the black-box inference-only paradigm. The central claim of the abstract is therefore unverifiable from the submitted artifact.

Significance. If the intended survey existed as described, it would be a potentially useful organizing contribution: a validated white/gray/black-box taxonomy for FedLLM could provide a common vocabulary for a rapidly growing literature and give visibility to inference-only approaches. However, in this submission there is nothing to evaluate. The actual body text is an unrelated ANN filtering survey with a different title, author list, and subject matter. No credit can be given for content that is absent, and the claimed contribution is not merely flawed but missing.

major comments (3)

[Abstract vs. Full Text] The abstract promises a comprehensive survey on federated tuning for LLMs and a white-box/gray-box/black-box taxonomy. The full text is a different paper: 'Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study' by Li et al., arXiv:2508.16263v2 [cs.DB]. The body contains no FedLLM content whatsoever, so the load-bearing premise of the submission is false.
[Body text (all sections)] No section of the body defines or even mentions the FedLLM taxonomy claimed in the abstract. The only taxonomy presented concerns Filtering ANN algorithms based on attribute types and filtering strategies. The claimed two-axis classification of FedLLM studies cannot be checked for correctness, completeness, or internal consistency because it does not exist in the manuscript.
[Survey methodology (claimed but absent)] Even taking the abstract at face value, no search strategy, inclusion criteria, or coverage analysis is provided for the claimed FedLLM survey. Comprehensiveness is a defining requirement for a survey, and the abstract alone cannot establish it. This is an additional missing-support issue, secondary to the full-text mismatch but relevant if the authors resubmit the intended paper.

minor comments (1)

[Metadata and formatting] The full text carries an ACM Reference Format block with a placeholder DOI (https://doi.org/XXXXXXX.XXXXXXX) and a ©2018 copyright line, which is inconsistent with the 2025 submission date. Such metadata must be corrected in any resubmission.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation is present because the submission's full text is a different paper; the claimed FedLLM survey/taxonomy has no body text, so there is no derivation chain to reduce to its inputs.

full rationale

The abstract promises a comprehensive survey of federated post-training of LLMs and a white-box/gray-box/black-box taxonomy. The supplied full text is instead 'Attribute Filtering in Approximate Nearest Neighbor Search: An In-depth Experimental Study' (arXiv:2508.16263v2), by different authors, on a different topic. There is no section defining the FedLLM taxonomy, no enumeration of method families, no search methodology, no completeness analysis, and no equations linking the taxonomy to the surveyed literature. Therefore no load-bearing step can be shown to be equivalent to its inputs by construction; there is no fitted parameter renamed as a prediction, no self-definitional taxonomy, and no self-citation chain to audit. The review rule requiring flagging of missing support applies: the central claim's preconditions (comprehensiveness and representativeness of the survey, and an adequate account of black-box inference-only FedLLM) are unverifiable from this artifact. That is a missing-support / integrity problem, not a circularity problem. The circularity score is 0, with the caveat that non-circularity cannot be affirmatively established for content that is absent.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No fitted numbers or physical postulates appear in the abstract; for a survey, the operative assumptions are about literature coverage and classification validity. The full text contains benchmark parameters (selectivity levels, datasets, index configurations), but those belong to arXiv:2508.16263v2, not to the paper the abstract describes, so they are not counted as parameters of the claimed survey. The taxonomy classes (white-box, gray-box, black-box) are organizational labels for existing method families, not newly postulated entities requiring falsifiable handles.

assumptions (3)

domain assumption The surveyed literature set is comprehensive and representative of the FedLLM field
The abstract claims comprehensiveness ('comprehensive survey'); completeness and representativeness are load-bearing for any taxonomy and cannot be verified because the survey body is absent (full text is a different paper).
domain assumption Model access and parameter efficiency are the two principal axes that meaningfully partition FedLLM methods
The abstract asserts this taxonomy structure without presenting evidence; if major method families (e.g., hybrid access regimes) fall outside both axes, the taxonomy misrepresents the field.
ad hoc to paper The white-box/gray-box/black-box trichotomy is the correct granularity for model access
The trichotomy is the paper's own organizing device; its adequacy as a partition of the literature is asserted in the abstract, not demonstrated by a taxonomy table or clustering of methods, neither of which is present in the submission.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View." pith.science (2026). https://pith.science/paper/UQZKIKSN

@misc{pith2026250816261,
  author       = {Pith},
  title        = {Pith review of: On the Evolution of Federated Post-Training Large Language Models: A Model Accessibility View},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UQZKIKSN}},
  note         = {Machine review of arXiv:2508.16261}
}

read the original abstract

Federated Learning (FL) enables training models across decentralized data silos while preserving client data privacy. Recent research has explored efficient methods for post-training large language models (LLMs) within FL to address computational and communication challenges. While existing approaches often rely on access to LLMs' internal information, which is frequently restricted in real-world scenarios, an inference-only paradigm (black-box FedLLM) has emerged to address these limitations. This paper presents a comprehensive survey on federated tuning for LLMs. We propose a taxonomy categorizing existing studies along two axes: model access-based and parameter efficiency-based optimization. We classify FedLLM approaches into white-box, gray-box, and black-box techniques, highlighting representative methods within each category. We review emerging research treating LLMs as black-box inference APIs and discuss promising directions and open challenges for future research.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 38 canonical work pages

[30]

Elias Jääsaari, Ville Hyvönen, and Teemu Roos. 2024. LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search.�� 37 (2024), 102121–102153

work page 2024
[31]

Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. 2019. Diskann: Fast accurate billion-point nearest neighbor search on a single node.�� 32 (2019)

work page 2019
[32]

Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search.�� 33, 1 (2010), 117–128

work page 2010
[33]

Hervé Jégou, Romain Tavenard, Matthijs Douze, and Laurent Amsaleg. 2011. Searching in one billion vectors: re-rank with source coding. In�� . IEEE, 861–864

work page 2011
[34]

Atsutake Kosuge and Takashi Oshima. 2019. An object-pose estimation accelera- tion technique for picking robot applications by using graph-reusing k-nn search. In�� . IEEE, 68–74

work page 2019
[35]

Joseph B Kruskal. 1956. On the shortest spanning subtree of a graph and the traveling salesman problem.�� 7, 1 (1956), 48–50

work page 1956
[36]

Jie Li, Haifeng Liu, Chuanghua Gui, Jianyu Chen, Zhenyuan Ni, Ning Wang, and Yuan Chen. 2018. The design and implementation of a real time visual search system on JD E-commerce platform. In�� . 9–16

work page 2018
[37]

Wen Li, Ying Zhang, Yifang Sun, Wei Wang, Mingjie Li, Wenjie Zhang, and Xuemin Lin. 2019. Approximate nearest neighbor search on high dimensional data—experiments, analyses, and improvement.�� 32, 8 (2019), 1475–1488

work page 2019

Show all 40 references

[38]

Anqi Liang, Pengcheng Zhang, Bin Yao, Zhongpu Chen, Yitong Song, and Guangxu Cheng. 2024. UNIFY: Unified Index for Range Filtered Approximate Nearest Neighbors Search.�� (2024)

2024
[39]

J MacQueen. 1967. Some methods for classification and analysis of multivariate observations. In��

1967
[40]

Yury Malkov, Alexander Ponomarenko, Andrey Logvinov, and Vladimir Krylov
[41]

Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs.�� 42, 4 (2018), 824–836

2018
[42]

Yusuke Matsui, Ryota Hinami, and Shin’ichi Satoh. 2018. Reconfigurable Inverted Index. In�� . 1715–1723

2018
[43]

Yitong Meng, Xinyan Dai, Xiao Yan, James Cheng, Weiwen Liu, Jun Guo, Benben Liao, and Guangyong Chen. 2020. Pmd: An optimal transportation-based user distance for recommender systems. In�� ...

2020
[44]

Jason Mohoney, Anil Pacaci, Shihabur Rahman Chowdhury, Ali Mousavi, Ihab F Ilyas, Umar Farooq Minhas, Jeffrey Pound, and Theodoros Rekatsinas. 2023. High- throughput vector similarity search in knowledge graphs.�� 1, 2 (2023), 1–25

2023
[45]

Lushuai Niu, Zhi Xu, Longyang Zhao, Daojing He, Jianqiu Ji, Xiaoli Yuan, and Mian Xue. 2023. Residual vector product quantization for approximate nearest neighbor search.�� 232 (2023), 120832

2023
[46]

Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. 2017. Embedding-based news recommendation for millions of users. In�� . 1933–1942

2017
[47]

James Jie Pan, Jianguo Wang, and Guoliang Li. 2024. Survey of vector database management systems.�� 33, 5 (2024), 1591–1615

2024
[48]

Zhibin Pan, Liangzhuang Wang, Yang Wang, and Yuchen Liu. 2020. Product quantization with dual codebooks for approximate nearest neighbor search. ��401 (2020), 59–68

2020
[49]

Rodrigo Paredes and Edgar Chávez. 2005. Using the k-nearest neighbor graph for proximity searching in metric spaces. In�� . Springe...

2005
[50]

Liana Patel, Peter Kraft, Carlos Guestrin, and Matei Zaharia. 2024. ACORN: Per- formant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data.�� 2, 3 (2024), 1–27

2024
[51]

Arkadiusz Paterek. 2007. Improving regularized singular value decomposition for collaborative filtering. In�� , Vol. 2007. 5–8

2007
[52]

Zhencan Peng, Miao Qiao, Wenchao Zhou, Feifei Li, and Dong Deng. [n. d.]. Dynamic Range-Filtering Approximate Nearest Neighbor Search. ([n. d.])
[53]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In�� ...

2021
[54]

Patrick Schäfer, Jakob Brand, Ulf Leser, Botao Peng, and Themis Palpanas. 2024. Fast and Exact Similarity Search in less than a Blink of an Eye.�� (2024)

2024
[55]

Chanop Silpa-Anan and Richard Hartley. 2008. Optimised KD-trees for fast image descriptor matching. In�� . IEEE, 1–8

2008
[56]

Godfried T Toussaint. 1980. The relative neighbourhood graph of a finite planar set.�� 12, 4 (1980), 261–268

1980
[57]

A Vaswani. 2017. Attention is all you need.�� (2017)

2017
[58]

Jianguo Wang, Xiaomeng Yi, Rentong Guo, Hai Jin, Peng Xu, Shengjun Li, Xi- angyu Wang, Xiangzhou Guo, Chengming Li, Xiaohai Xu, et al. 2021. Milvus: A purpose-built vector data management system. In�� . 2614–2627

2021
[59]

Mengzhao Wang, Lingwei Lv, Xiaoliang Xu, Yuxiang Wang, Qiang Yue, and Jiongkang Ni. 2024. An efficient and robust framework for approximate near- est neighbor search with attribute constraint.�� 36 (2024)

2024
[60]

Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. 2021. A com- prehensive survey and experimental comparison of graph-based approximate nearest neighbor search.�� (2021)

2021
[61]

Chuangxian Wei, Bin Wu, Sheng Wang, Renjie Lou, Chaoqun Zhan, Feifei Li, and Yuanzhe Cai. 2020. AnalyticDB-V: a hybrid analytical engine towards query fusion for structured and unstructured data.�� 13, 12 (2020), 3152–3165

2020
[62]

Yuexuan Xu, Jianyang Gao, Yutong Gou, Cheng Long, and Christian S Jensen
[63]

Shuo Yang, Jiadong Xie, Yingfan Liu, Jeffrey Xu Yu, Xiyue Gao, Qianru Wang, Yanguo Peng, and Jiangtao Cui. 2024. Revisiting the Index Construction of Proximity Graph-Based Approximate Nearest Neighbor Search.�� (2024)

2024
[64]

Wen Yang, Tao Li, Gai Fang, and Hong Wei. 2020. Pase: Postgresql ultra-high- dimensional approximate nearest neighbor search extension. In�� . 2241–2253

2020
[65]

Qianxi Zhang, Shuotao Xu, Qi Chen, Guoxin Sui, Jiadong Xie, Zhizhen Cai, Yaoqi Chen, Yinxuan He, Yuqing Yang, Fan Yang, et al . 2023. �VBASE�: Unifying Online Vector Similarity Search and Relational Queries via Relaxed Monotonicity. In�� ...

2023
[66]

Weijie Zhao, Shulong Tan, and Ping Li. 2022. Constrained approximate similarity search on proximity graph.�� (2022)

2022
[67]

Chaoji Zuo, Miao Qiao, Wenchao Zhou, Feifei Li, and Dong Deng. 2024. SeRF: Segment Graph for Range-Filtering Approximate Nearest Neighbor Search.�� 2, 1 (2024), 1–26. Received 13 March 2025; revised 1 July 2025; accepted 24 August 2025

2024
[2014]

Approximate nearest neighbor algorithm based on navigable small world graphs.�� 45 (2014), 61–68

2014
[2024]

iRangeGraph: Improvising Range-dedicated Graphs for Range-filtering Nearest Neighbor Search.�� 2, 6 (2024), 1–26

2024

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [30]

Elias Jääsaari, Ville Hyvönen, and Teemu Roos. 2024. LoRANN: Low-Rank Matrix Factorization for Approximate Nearest Neighbor Search.�� 37 (2024), 102121–102153

work page 2024

[2] [31]

Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. 2019. Diskann: Fast accurate billion-point nearest neighbor search on a single node.�� 32 (2019)

work page 2019

[3] [32]

Herve Jegou, Matthijs Douze, and Cordelia Schmid. 2010. Product quantization for nearest neighbor search.�� 33, 1 (2010), 117–128

work page 2010

[4] [33]

Hervé Jégou, Romain Tavenard, Matthijs Douze, and Laurent Amsaleg. 2011. Searching in one billion vectors: re-rank with source coding. In�� . IEEE, 861–864

work page 2011

[5] [34]

Atsutake Kosuge and Takashi Oshima. 2019. An object-pose estimation accelera- tion technique for picking robot applications by using graph-reusing k-nn search. In�� . IEEE, 68–74

work page 2019

[6] [35]

Joseph B Kruskal. 1956. On the shortest spanning subtree of a graph and the traveling salesman problem.�� 7, 1 (1956), 48–50

work page 1956

[7] [36]

Jie Li, Haifeng Liu, Chuanghua Gui, Jianyu Chen, Zhenyuan Ni, Ning Wang, and Yuan Chen. 2018. The design and implementation of a real time visual search system on JD E-commerce platform. In�� . 9–16

work page 2018

[8] [37]

Wen Li, Ying Zhang, Yifang Sun, Wei Wang, Mingjie Li, Wenjie Zhang, and Xuemin Lin. 2019. Approximate nearest neighbor search on high dimensional data—experiments, analyses, and improvement.�� 32, 8 (2019), 1475–1488

work page 2019