Pith. sign in

REVIEW 4 major objections 5 minor 19 references

LLM Agent for Hyper-Parameter Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A profile-guided LLM agent iteratively tunes WS-PSO-CM hyper-parameters and, within six iterations, raises the minimal sum-rate from hand-tuned and random baselines to 22.28 bps/Hz.

desk verdict A workmanlike LLM-agent-for-HPO pipeline whose headline gains rest on a single unseeded run; the idea is modest and the evidence doesn't hold up yet. read the letter →

arxiv 2506.15167 v2 pith:6UP5UDUZ submitted 2025-06-18 cs.IT cs.AImath.IT

Wanzhe Wang , Jianqiu Peng , Menghao Hu , Weihuang Zhong , Tong Zhang , Shuai Wang , Yixin Zhang , Mingjie Shao
show 1 more author
Wanli Ni
This is my paper · ORCID
classification cs.ITcs.AImath.IT
keywords largelanguagemodelcontextprotocolradiomapunmannedaerialvehiclehyper-parameteroptimizationWS-PSO-CMparticleswarmUAVtrajectoryandcommunication
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes an LLM agent that automatically tunes the eight hyper-parameters of the WS-PSO-CM algorithm, a warm-started particle swarm optimizer with crossover and mutation used for radio-map-enabled UAV trajectory and communication. It claims that, within six iterations, the agent finds settings that achieve a minimal sum-rate of 22.28 bps/Hz, which is 54.33% higher than the hand-tuned heuristic configuration and 72.61% higher than a uniform-random configuration. The reason this matters is that hyper-parameter selection for such optimizers is currently manual and expertise-heavy; an automated feedback loop that reads the algorithm's fitness output could replace that labor and transfer to other optimization algorithms.

What carries the argument

The load-bearing mechanism is the closed exploration loop around the WS-PSO-CM solver. WS-PSO-CM is a particle swarm optimizer that initializes particles with a convex-optimization warm start, then applies crossover and mutation to maintain diversity, with a fitness function that weights the minimal sum-rate against speed, turning-angle, and building-collision penalties. The LLM agent proposes the eight tunable hyper-parameters (particle count, inertia weight, cognitive and social coefficients, and the four fitness weights), receives the solver's fitness outcome through a Model Context Protocol tool server, and revises its proposal until the terminal condition triggers.

What would settle it

Run the same WS-PSO-CM solver on the same campus scenario with the LLM-recommended settings and each baseline setting across many random seeds; if the averaged minimal sum-rates overlap within their standard deviations, or if the baseline matches 22.28 bps/Hz once given the same particle count and compute budget, the claimed 54.33% and 72.61% gains are not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that a profile-guided LLM agent can navigate the hyper-parameter space of WS-PSO-CM by an iterative propose-and-observe loop, without any hand-tuning. The agent's profile fixes the hyper-parameter boundaries, the objective (maximizing the minimal sum-rate), the stopping condition, and a conservative or aggressive optimization strategy; a Model Context Protocol server hands each proposed set to the solver, and the solver's returned minimal sum-rate is fed back as the signal for the next proposal. Across six iterations the agent's proposals improve the minimal sum-rate from 16.33 to 22.28 bps/Hz, compared with 14.44 bps/Hz for the human-heuristic baseline and 12.91 bps/Hz for the uniform-random baseline. The paper interprets this as evidence that LLM agents with PSO and WS-PSO-CM knowledge can act as general high-performance hyper-parameter seekers.

Load-bearing premise

The paper's conclusion depends on the assumption that the reported single-run results on one campus scenario are representative enough to attribute the higher minimal sum-rate to the LLM agent's tuning rather than to run-to-run randomness or unequal compute budgets.

Editorial extensions

If this is right

  • Six iterations of LLM-guided proposals suffice to raise the reported minimal sum-rate from 16.33 bps/Hz to 22.28 bps/Hz in the tested scenario.
  • The LLM-recommended particle counts (40 to 50) outperform a human heuristic that uses 100 particles, so the automated settings also reduce per-run compute.
  • Hyper-parameter tuning for WS-PSO-CM can in principle be fully automated, needing only an algorithm description, a fitness signal, and a terminal condition.
  • The same profile-and-tool structure applies to other tuning-hungry optimizers, including multi-UAV coordination, resource allocation, radio map construction, and reinforcement learning tasks, as the paper's conclusion states.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported 54.33% and 72.61% gains compare against one uniform-random configuration and one heuristic configuration on a single scenario; a natural test extension is to average over many seeds and over several radio-map scenarios before treating the percentages as general.
  • Because the human baseline uses 100 particles while the LLM proposals cap at 50, an equal-compute comparison would clarify how much of the gain comes from tuning quality rather than from a smaller, cheaper search.
  • The same feedback loop could be extended to tune not only the hyper-parameters but also the penalty weights and constraint parameters of problem (P0), effectively letting the agent reshape the optimizer's objective during search.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper proposes an LLM agent for tuning the hyper-parameters of the WS-PSO-CM algorithm used in radio map-enabled UAV trajectory and communication optimization. The agent is constructed with a Profile that specifies hyper-parameter boundaries, task objective, terminal condition, strategy, and LLM configuration, and it uses the Model Context Protocol (MCP) to interact with two servers: one passing hyper-parameters to numerical software and one managing history. In each iteration, the LLM proposes a hyper-parameter set, WS-PSO-CM is executed, and the resulting minimal sum-rate is fed back to the LLM. Experiments on a 240x400x60 m^3 HITSZ campus scenario with 4 UAVs and 8 UGVs report a minimal sum-rate of 22.28 bps/Hz after six LLM iterations, compared with 14.44 bps/Hz for the human heuristic configuration from the authors' prior work [3] and 12.91 bps/Hz for one uniformly randomly generated configuration, corresponding to claimed gains of 54.33% and 72.61%.

Significance. If the performance gain were established with proper statistical control, the paper would be a useful demonstration of LLM agents for hyper-parameter optimization in wireless communication, with a concrete MCP-based tool integration and an iterative feedback loop. The paper explicitly provides the algorithm description, the profile concept, and the iteration-level hyper-parameter table, which are helpful for replication. However, the central empirical claim is currently supported only by single-run scalar comparisons on one scenario, with no repeated trials, seeds, confidence intervals, or standard HPO baselines; as it stands, the evidence is not sufficient to conclude that the LLM agent significantly outperforms either baseline. The contribution is therefore promising but not yet established.

major comments (4)
  1. [Section IV, Figs. 4 and 5] Every reported minimal sum-rate is a single number, with no seeds, repeated trials, confidence intervals, or significance tests. WS-PSO-CM is stochastic through the random variables rand1 and rand2 in Eq. (4a) and through randomized initialization, crossover, and mutation; the LLM itself (DeepSeek R1, with unspecified temperature) is also stochastic. Consequently the claimed 54.33% and 72.61% gains may be within the run-to-run variation of the stochastic optimizer. The paper should repeat each configuration over many seeds, report mean and variance or boxplots, and perform a statistical test before claiming significance.
  2. [Section IV, baselines] The comparison is not controlled. The human heuristic baseline uses Pnum=100, whereas every LLM proposal in Table I uses Pnum<=50, so the comparison mixes hyper-parameter quality with a different particle budget. The 'uniform distribution' baseline is a single sampled configuration (Pnum=58, w=0.8765, c1=5.4321, etc.), not a distribution; without defining the sampling ranges and drawing multiple random configurations, the 72.61% gain is not a statement about random generation as a method. The authors should report the hyper-parameter bounds used in the profile, draw a full random baseline with the same number of evaluations, and match computational budgets.
  3. [Section IV, paragraph beginning 'Since we only consider...'] The authors explicitly exclude Grid Search and Bayesian Optimization because they consider only a few iterations. Since the paper claims automatic hyper-parameter tuning for WS-PSO-CM and frames current methods as heuristic-based, the absence of any standard HPO baseline makes the broader claim untested. At minimum, a random search with the same number of WS-PSO-CM evaluations and a simple Bayesian optimization baseline should be included to position the LLM agent relative to established methods.
  4. [Sections III-B2 and IV] The profile is said to specify the boundary of hyper-parameters and LLM configurations, but the paper never reports these boundaries, the DeepSeek R1 model version, temperature, prompt, or the terminal condition. Without these details the experiment is not reproducible and the reader cannot know whether the uniform baseline draws from the same feasible region. Please provide the full profile and LLM settings.
minor comments (5)
  1. [Abstract and Section IV] The phrases 'an Large Language Model' and 'an uniform distribution' should be corrected to 'a Large Language Model' and 'a uniform distribution'.
  2. [Equation (4a)] Equation (4a) updates velocity as Vk = omega*Vk + c1*rand1*(pBestk - Vk) + ..., but the second term should almost certainly be (pBestk - Qk), the difference between the personal best and the current position, rather than the current velocity. Please correct the formula or define the notation.
  3. [Algorithm 1] Algorithm 1 line 5 writes 'rhoBestk' while the text uses 'pBestk'; please unify the notation.
  4. [Section II-A and Section IV] There is a duplicated phrase 'scheduling scheduling' in Section II-A, and the list of UGV speeds in Section IV uses inconsistent comma separators.
  5. [Figure 4] The y-axis label 'minimal sum-rate' is ambiguous because (3a) maximizes the minimum over UGVs; please clarify whether the plotted quantity is the optimized max-min value.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: LLM-proposed hyperparameters are evaluated by an independent simulator, and the self-cited baseline is not load-bearing.

full rationale

The paper's central claim is empirical: an LLM agent iteratively proposes hyperparameters for WS-PSO-CM, and the algorithm's simulator returns a minimal sum-rate that is compared against baselines. There is no derivation chain in which an output is equivalent to an input by construction. The LLM agent does not fit a parameter to the reported metric and then relabel that fit as a prediction; each proposed hyperparameter set in Table I is evaluated through the independently coded WS-PSO-CM objective, so the reported 22.28 bps/Hz is a simulation output, not a restatement of the prompt or profile. The human-heuristic baseline is taken from the authors' prior paper [3], which is a self-citation, but it is used only as a comparison point and does not justify the LLM agent's performance; the comparison would remain meaningful even if [3] were authored by someone else. No uniqueness theorem, ansatz, or definition is imported from the authors' prior work to force the conclusion. The main weaknesses are statistical and experimental design issues: single runs without seeds or error bars, the human baseline using Pnum=100 while LLM proposals use at most Pnum=50, and only one uniform-random draw. These are correctness risks about whether the claimed gains are reproducible and attributable to hyperparameter quality, not circularity. Therefore the paper does not exhibit self-definitional reasoning, fitted-input-called-prediction, or any other circularity pattern.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper contributes an engineering pipeline, not a derivation. Its central claim rests on the imported WS-PSO-CM algorithm and scenario from [3], on an unspecified LLM profile and prompt, and on an implicit assumption that single runs are representative. The three free parameters listed capture the most consequential unspecified design choices.

free parameters (3)
  • LLM iteration count / terminal condition = 6 iterations
    The agent stops after six iterations by design; the paper does not report convergence or sensitivity to stopping at other points, and the final gain is evaluated at this chosen stop.
  • Hyperparameter search boundaries in LLM profile
    The profile sets boundaries for Pnum, omega, c1, c2, and k1-k4, but the exact ranges are not disclosed, so the search space that made the result possible is unknown.
  • LLM configuration (model version, temperature, prompt) = DeepSeek R1, details undisclosed
    The LLM's sampling behavior depends on these settings; without them the agent's exploration cannot be reproduced.
assumptions (5)
  • domain assumption WS-PSO-CM as defined in [3] is correct and appropriate for the radio-map UAV problem.
    The paper imports the algorithm, its fitness function, and the simulation setup from [3] without re-deriving them, so any flaw in [3] propagates into the comparison.
  • ad hoc to paper The single HITSZ campus scenario with 8 UGVs and 4 UAVs is representative.
    All results come from one digital-twin environment; no other radio map or geometry is tested.
  • ad hoc to paper One run per hyperparameter set is a sufficient estimator of the stochastic objective.
    No seeds, trials, or confidence intervals are reported for the PSO/LLM loop.
  • ad hoc to paper The LLM's suggested hyperparameters are caused by its reasoning about PSO knowledge, not by chance or memorization.
    No control experiment (for example, random LLM suggestions or a fixed random search with the same budget) is provided.
  • domain assumption MCP servers deliver algorithm outputs and parameter updates reliably.
    The framework assumes the MCP tool invocation mechanism works as described, though this is not the main source of uncertainty.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LLM Agent for Hyper-Parameter Optimization." pith.science (2026). https://pith.science/paper/6UP5UDUZ

@misc{pith2026250615167,
  author       = {Pith},
  title        = {Pith review of: LLM Agent for Hyper-Parameter Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6UP5UDUZ}},
  note         = {Machine review of arXiv:2506.15167}
}
read the original abstract

Hyper-parameters are essential and critical for the performance of communication algorithms. However, current hyper-parameters optimization approaches for Warm-Start Particles Swarm Optimization with Crossover and Mutation (WS-PSO-CM) algorithm, designed for radio map-enabled unmanned aerial vehicle (UAV) trajectory and communication, are primarily heuristic-based, exhibiting low levels of automation and improvable performance. In this paper, we design an Large Language Model (LLM) agent for automatic hyper-parameters-tuning, where an iterative framework and Model Context Protocol (MCP) are applied. In particular, the LLM agent is first set up via a profile, which specifies the boundary of hyper-parameters, task objective, terminal condition, conservative or aggressive strategy of optimizing hyper-parameters, and LLM configurations. Then, the LLM agent iteratively invokes WS-PSO-CM algorithm for exploration. Finally, the LLM agent exits the loop based on the terminal condition and returns an optimized set of hyperparameters. Our experiment results show that the minimal sum-rate achieved by hyper-parameters generated via our LLM agent is significantly higher than those by both human heuristics and random generation methods. This indicates that an LLM agent with PSO and WS-PSO-CM algorithm knowledge is useful in seeking high-performance hyper-parameters.

Figures

Figures reproduced from arXiv: 2506.15167 by the authors.

Figure 1
Figure 1. LLM agent modules and functionalities. action plan. Alternatively, planning with feedback integrates feedback from the environment, human input, or the model itself, allowing the agent to refine its plan and adjust its approach dynamically in response to changing conditions. 4) Action: The Action functionality involves the execution of the agent’s formulated plans. The action target determines the target of the acti… view at source ↗
Figure 3
Figure 3. The framework of proposed LLM agent. TABLE I: Hyper-Parameters during Iterations Index Pnum k1 k2 k3 k4 w c1 c2 1 46 0.12 0.65 0.15 0.06 0.68 1.55 1.45 2 50 0.1 0.7 0.1 0.1 0.729 1.494 1.494 3 40 0.1 0.7 0.1 0.05 0.7 1.5 1.5 4 50 0.15 0.6 0.3 0.1 0.6 1.8 1.8 5 40 0.15 0.6 0.2 0.05 0.65 1.6 1.4 6 40 0.12 0.63 0.2 0.02 0.68 1.7 1.3 • Baseline 1 (Human Heuristics in [3]): These hyper￾parameters are used in [3] based on… view at source ↗
Figure 4
Figure 4. Minimal sum-rate over iterations by LLM agent. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Performance comparison against baselines. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages

  1. [1]

    Energy efficient resource allocation in UAV-enabled mobile edge computing networks,

    Z. Yang, C. Pan, K. Wang, and M. Shikh-Bahaei, “Energy efficient resource allocation in UAV-enabled mobile edge computing networks,” IEEE Trans. Wireless Commun., vol. 18, no. 9, pp. 4576–4589, 2019

    work page 2019

  2. [2]

    Integrated robotics networks with co-optimization of drone placement and air-ground communications,

    M. Hu, T. Zhang, S. Wang, G. Li, Y. Chen, Q. Li, and G. Chen, “Integrated robotics networks with co-optimization of drone placement and air-ground communications,” in Proc. VTC-Fall, pp. 1–5, 2023

    work page 2023

  3. [4]

    Deep learning-based radio map for MIMO-OFDM downlink precoding,

    W. Wang, B. Yang, and W. Zhang, “Deep learning-based radio map for MIMO-OFDM downlink precoding,” J. Commun. Inform. Netw., vol. 8, no. 3, pp. 203–211, 2023

    work page 2023

  4. [5]

    Radio map assisted path planning for UAV anti-jamming communications,

    Y. Dong, C. He, Z. Wang, and L. Zhang, “Radio map assisted path planning for UAV anti-jamming communications,”IEEE Signal Process. Lett., vol. 29, pp. 607–611, 2022. Fig. 6: Visualization of the WS-PSO-CM optimized UAV 3D trajectories by proposed hyper-parameters

    work page 2022

  5. [6]

    Radio map-based 3D path planning for cellular-connected UAV,

    S. Zhang and R. Zhang, “Radio map-based 3D path planning for cellular-connected UAV,”IEEE Trans. Wireless Commun., vol. 20, no. 3, pp. 1975–1989, 2021

    work page 1975

  6. [7]

    Radio map-based beamforming assisted with reduced pilots,

    B. Yang, W. Wang, and W. Zhang, “Radio map-based beamforming assisted with reduced pilots,” IEEE Trans. Wireless Commun., pp. 1–1, 2025

    work page 2025

  7. [8]

    Radio-map-based UAV placement design for UAV-assisted relaying networks,

    X. Yuan, Y. Hu, J. Gross, and A. Schmeink, “Radio-map-based UAV placement design for UAV-assisted relaying networks,” in Proc. IEEE SSP, pp. 286–290, 2021

    work page 2021

  8. [9]

    Dynamic object tracking by multi- UAV with time-variant radio maps,

    Y. Dong, C. He, and Z. J. Wang, “Dynamic object tracking by multi- UAV with time-variant radio maps,” IEEE Trans. Wireless Commun. , vol. 23, no. 7, pp. 7471–7487, 2024

    work page 2024

Show all 19 references
  1. [10]

    Joint power and 3D trajectory optimization for UAV-enabled wireless powered commu- nication networks with obstacles,

    H. Pan, Y. Liu, G. Sun, J. Fan, S. Liang, and C. Yuen, “Joint power and 3D trajectory optimization for UAV-enabled wireless powered commu- nication networks with obstacles,” IEEE Trans. Commun., vol. 71, no. 4, pp. 2364–2380, 2023

    2023

  2. [11]

    LLM-based agentic systems in medicine and healthcare,

    J. Qiu, K. Lam, G. Li, A. Acharya, T. Y. Wong, A. Darzi, W. Yuan, and E. J. Topol, “LLM-based agentic systems in medicine and healthcare,” Nature Mach. Intell., vol. 6, no. 12, pp. 1418–1420, 2024

    2024

  3. [12]

    Mathematical discoveries from program search with large language models,

    B. Romera-Paredes, M. Barekatain, A. Novikov, M. Balog, M. P. Kumar, E. Dupont, F. J. Ruiz, J. S. Ellenberg, P. Wang, O. Fawzi, et al. , “Mathematical discoveries from program search with large language models,” Nature, vol. 625, no. 7995, pp. 468–475, 2024

    2024

  4. [13]

    Industrial internet of things with large language models (LLMs): an intelligence- based reinforcement learning approach,

    Y. Ren, H. Zhang, F. R. Yu, W. Li, P. Zhao, and Y. He, “Industrial internet of things with large language models (LLMs): an intelligence- based reinforcement learning approach,” IEEE Trans. Mobile Comput. , 2024

    2024

  5. [14]

    Federated low-rank adaptation for large models fine-tuning over wireless networks,

    H. Sun, H. Tian, W. Ni, J. Zheng, D. Niyato, and P. Zhang, “Federated low-rank adaptation for large models fine-tuning over wireless networks,” IEEE Trans. Wireless Commun., vol. 24, no. 1, pp. 659–675, 2025

    2025

  6. [15]

    Federated intelligence: When large AI models meet federated fine-tuning and collaborative reasoning at the network edge,

    W. Ni, H. Sun, H. Ao, and H. Tian, “Federated intelligence: When large AI models meet federated fine-tuning and collaborative reasoning at the network edge,” arXiv preprint arXiv:2503.21412, 2025

    2025 arXiv

  7. [16]

    Large language model-based wireless network design,

    K. Qiu, S. Bakirtzis, I. Wassell, H. Song, J. Zhang, and K. Wang, “Large language model-based wireless network design,” IEEE Wireless Commun. Lett., 2024

    2024

  8. [17]

    Maestro: LLM-driven collaborative automation of intent-based 6G networks,

    I. Chatzistefanidis, A. Leone, and N. Nikaein, “Maestro: LLM-driven collaborative automation of intent-based 6G networks,”IEEE Netw. Lett., vol. 6, no. 4, pp. 227–231, 2024

    2024

  9. [18]

    Large language models (LLMs) assisted wireless network deployment in urban settings,

    N. Sevim, M. Ibrahim, and S. Ekin, “Large language models (LLMs) assisted wireless network deployment in urban settings,” in Proc. IEEE VTC-Fall, pp. 1–7, 2024

    2024

  10. [19]

    Large language model agents for radio map generation and wireless network planning,

    H. Quan, W. Ni, T. Zhang, X. Ye, Z. Xie, S. Wang, Y. Liu, and H. Song, “Large language model agents for radio map generation and wireless network planning,” IEEE Netw. Lett., pp. 1–1, 2025

    2025

  11. [2025]

    Available: https://arxiv.org/abs/2505.06944

    [Online]. Available: https://arxiv.org/abs/2505.06944

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.