Pith. sign in

REVIEW 5 major objections 5 minor 60 references

SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation

T0 review · 5 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that fine-tuning a pretrained model by updating only the highest-energy Hartley coefficients of its weights matches or beats LoRA and FourierFT with a fraction of the trainable parameters.

desk verdict SSH is a solid, incremental PEFT paper with a genuinely new DHT-based variant, but it omits the one baseline—the authors' own SDCT—that would actually isolate what DHT buys you. read the letter →

arxiv 2502.05539 v1 pith:LXV2JR7L submitted 2025-02-08 cs.CV cs.LG

Yixian Shen , Qi Bi , Jia-Hong Huang , Hongyi Zhu , Andy D. Pimentel , Anuj Pathania This is my paper · ORCID
classification cs.CVcs.LG
keywords parameter-efficientfine-tuningdiscreteHartleytransformsparsespectrumadaptationspectralcoefficientselectionlow-ranklargelanguagemodelsGLUEbenchmarkvisiontransformers
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes SSH, a parameter-efficient fine-tuning method that learns updates in the discrete Hartley spectral domain rather than in weight space. The claim is that selecting the largest-energy Hartley coefficients of the pretrained weights and updating only those, then projecting back with the inverse Hartley transform, yields accuracy at or above LoRA and FourierFT while using dramatically fewer trainable parameters. If true, fine-tuning large models would no longer need low-rank matrix factors or complex Fourier arithmetic; a real-valued, self-inverse transform would suffice. The paper supports this with GLUE, E2E, instruction-tuning, summarization, GSM8K, and image-classification results.

What carries the argument

The load-bearing object is the two-dimensional discrete Hartley transform of each pretrained weight matrix, a real-valued transform whose inverse is the same operation as the forward transform. The selection rule computes per-coefficient energy $E(u,v)=|H(u,v)|^2$, keeps the top energy-ranked coefficients together with a randomly chosen remainder, and trains only those selected coefficients per layer; gradients are masked inside the spectral domain and the inverse transform maps the updates back into weight space.

What would settle it

Run SSH on the GLUE benchmark with $\delta=0$, so all selected Hartley coefficients are chosen uniformly at random rather than by energy; if the average accuracy matches or beats the reported $\delta=0.7$ result, the paper's central claim that high-energy coefficients carry task-relevant information is falsified.

Watch

Extended reading notes

Core claim

SSH's central discovery is that a pretrained weight matrix's discrete Hartley transform concentrates the information needed for task adaptation in a small set of high-energy spectral coefficients, and updating only those coefficients—chosen per layer by energy, with a random remainder for diversity—produces weight updates that match or exceed existing PEFT methods. Because the Hartley transform is real-valued and equal to its own inverse, the forward-backward projections avoid the complex arithmetic of FourierFT and its associated overhead. Concretely, the paper reports the best GLUE average for RoBERTa-base with 0.018M trainable parameters and for RoBERTa-large with 0.036M, and reductions of up to 55% in GFLOPs relative to FourierFT.

Load-bearing premise

The method assumes that the few largest frequency-like Hartley components of the pretrained weights are the parts that most need changing for a new task, and that the same fixed number of components works for every layer.

Editorial extensions

If this is right

  • If SSH's results hold, parameter-efficient fine-tuning can be done with a real-valued transform whose forward and inverse are the same routine, eliminating the separate complex-valued path used by FourierFT.
  • Memory for gradients and optimizer states would scale with the number of selected spectral coefficients per layer rather than with the layer's full dimensions, making larger models cheaper to adapt.
  • The reported GLUE averages (85.46 on RoBERTa-base and 88.17 on RoBERTa-large) with the fewest trainable parameters among compared methods imply the energy-selection heuristic is doing real work, not just saving parameters.
  • Instruction tuning on LLaMA2-7B, LLaMA2-13B, and LLaMA3.1-8B with roughly 0.05M trainable parameters produces GPT-4-judged scores at or near full fine-tuning, suggesting the approach scales to billion-parameter models.
  • On image classification, SSH matches full fine-tuning on EuroSAT and OxfordPets with 54K trainable parameters on ViT-B, indicating the method transfers beyond language tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Our inference: because DHT is real-valued and self-inverse, SSH should port directly to convolutional layers and other non-matrix parameter tensors, a regime the paper does not test.
  • Our inference: the energy-ratio ablation varies delta while fixing the per-layer budget, so a layer-wise allocation of coefficients, with more budget where adaptation signal is strongest, would likely improve on the fixed budget used in the paper.
  • Our inference: the paper's compute advantage is stated in GFLOPs; a direct wall-clock comparison on identical hardware would test whether the theoretical savings translate to end-to-end training time, including the one-off DHT cost the paper lists as a limitation.
  • Our inference: comparing SSH against a discrete cosine transform variant with the same selection rule would isolate whether Hartley's self-inverse symmetry, rather than real-valuedness alone, explains the gains over FourierFT.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. SSH proposes a parameter-efficient fine-tuning method that applies the 2D discrete Hartley transform (DHT) to pretrained weight matrices, selects a small budget n of spectral coefficients per layer using an energy heuristic together with random sampling, and learns only those coefficients while mapping updates back through the inverse DHT. The paper evaluates SSH on GLUE with RoBERTa-base/large, E2E generation with GPT-2, instruction tuning on LLaMA-2 and LLaMA-3.1, text summarization with BART-Large, GSM8K mathematical reasoning, and image classification with ViT-B/L, comparing against LoRA, DoRA, VeRA, FourierFT, AFLoRA, LaMDA, adapter methods, and full fine-tuning. The central claims are that SSH outperforms existing PEFT methods while using fewer trainable parameters and reducing GFLOPs relative to FourierFT.

Significance. If the claims hold, SSH would be a useful addition to the spectral-PEFT family, offering strong parameter efficiency and competitive accuracy across multiple modalities. The paper's parameter-count accounting is transparent and consistent with the reported n-per-matrix values, and the appendix hyperparameter tables are sufficiently detailed to reproduce the experiments. The breadth of evaluation is a clear strength, as is the explicit algorithmic description. However, the paper does not yet establish the specificity of its central contribution: the DHT-based selection is not compared against the authors' own DCT-based SDCT method, and the energy-selection ablation lacks a pure-random-selection control. Missing variance information and an unverified GFLOP claim further weaken the empirical and computational conclusions. The contribution is plausible but conditional on these comparisons.

major comments (5)
  1. [§4.1, Tables 2–5] The closest prior method, SDCT (Shen et al., 2024b, arXiv:2410.09103), appears in the reference list but is omitted from the baseline list in Section 4.1 and from every comparison table. SDCT is the direct predecessor: it selects a sparse set of real spectral coefficients of the pretrained weights and learns them under an inverse real transform. Because the paper's stated novelty over FourierFT is the use of a real, symmetric transform, a head-to-head SSH–SDCT comparison (or a DCT variant of the same pipeline) is needed to attribute the observed gains to DHT rather than to the already-published DCT-based sparse-spectral mechanism. Without it, the claim that SSH outperforms existing PEFT methods is not established at the level that isolates the actual contribution.
  2. [§4.8, Eq. (3)] The energy-ratio ablation in Figure 4 varies δ only over {0.5, 0.6, 0.7, 0.8, 0.9} and never includes δ = 0 (pure random spectral selection) or δ = 1 (pure energy selection). Since the energy-based selection rule in Eq. (3) is one of the paper's two stated contributions, this ablation cannot distinguish the contribution of energy-based selection from random spectral selection. The authors should add δ = 0 and δ = 1, and ideally compare against a fixed-budget random-selection baseline with the same n, before claiming that the energy-compaction heuristic is responsible for the results.
  3. [Tables 2, 4, and 5] GLUE results are reported as medians of five runs with no standard deviations, and several margins over the next-best method are small (e.g., 85.46 vs. 85.42 on the RoBERTa-base average; 88.17 vs. 88.03 on RoBERTa-large; 7.71 vs. 7.67 on Vicuna for LLaMA3.1-8B). The instruction-tuning scores in Table 4 are GPT-4 judgments without variance. Without per-seed results, standard deviations, or a paired significance test, the claims that SSH 'consistently delivers' and 'outperforms' baselines are not quantitatively supported. Please report mean ± std or per-seed values for all runs.
  4. [§1, Fig. 1] The headline computational-efficiency claim of '55% fewer GFLOPs compared to FourierFT' is not supported by any description of how the GFLOP counts were obtained. No measurement methodology, operation decomposition, or runtime experiment is given, and the one-time DHT overhead acknowledged in Section 5 is not included in the comparison. Since computational efficiency is a stated contribution, the paper should report either a precise FLOP-counting procedure (including forward and backward passes) or measured training/inference time and peak memory.
  5. [§3.1, Eq. (2)] Equation (2) defines the Hartley kernel as cos(2πux/d1 + 2πvy/d2) − sin(2πux/d1 + 2πvy/d2), whereas the standard discrete Hartley transform uses cas(θ) = cos θ + sin θ. With the minus sign the transform is not the standard DHT, and the claimed self-inverse property used in Algorithm 1 and the Introduction does not follow. Please correct Eq. (2) to the standard DHT definition or justify the nonstandard kernel and prove the required inverse and symmetry properties.
minor comments (5)
  1. [§4.3] The sentence 'all with 10.3% fewer parameters than LoRA' is numerically incorrect for the reported SSH count of 0.036M versus LoRA's 0.35M on GPT-2 Medium; this is 89.7% fewer parameters (or 10.3% of LoRA's parameter count).
  2. [§4.2] The phrase '7682 available spectral coefficients' should be '768^2 available spectral coefficients', and the corresponding '10242' should be '1024^2'; the superscript formatting is lost and makes the sentence hard to read.
  3. [§5] The limitation heading 'One-go Additional Computational overhead' contains wording and capitalization errors; it should read 'One-time additional computational overhead'.
  4. [Algorithm 1] The line 'Set n frequencies ← requireGrad(True)' is ambiguous; it should state 'Set the selected n frequencies to requireGrad(True) and freeze the remaining d1 × d2 − n coefficients.'
  5. [Table 1] The columns in Table 1 appear misaligned, especially the repeated '200' entries for SSH; please reformat the table so that the SSH parameter n and the corresponding parameter count are unambiguously matched.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the method is an empirical benchmark with a standard masked reparameterization, and the closest prior SDCT baseline is omitted from comparisons, which is a novelty gap rather than a construction-level circular step.

full rationale

This is an empirical PEFT benchmark paper. The claimed results are task-accuracy measurements obtained by training the selected spectral coefficients against the task loss, not quantities derived from the method's own equations. The selection mask in Eq. (3) is computed from the pretrained weights only, so no label-dependent quantity is fitted and then re-reported as a prediction. The forward and backward definitions in Eqs. (5)-(7) are a standard linear reparameterization with a mask; there is no self-referential loop in which an output is defined as its own input. The only self-referential element is the citation of the authors' own SDCT prior work (Shen et al., 2024b, arXiv:2410.09103) in the reference list, which is omitted from the baseline list in Section 4.1. SDCT is the closest prior method, and its absence means the paper never isolates whether DHT or the shared sparse-real-spectrum mechanism drives the observed gains. That is a missing comparison affecting the novelty claim, not a construction-level circularity. Similarly, the energy-ratio ablation in Fig. 4 does not include delta=0 (pure random selection), so the benefit of energy-based selection over random selection is not directly demonstrated; again, this is an experimental gap, not a circular reduction. The paper is self-contained against external benchmarks including LoRA, FourierFT, DoRA, VeRA, AFLoRA, and LaMDA, so the appropriate circularity score is low.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

No new physical or architectural entities are introduced; DHT is a standard mathematical transform. The method is a training procedure, not a new model component. The main free parameters are the per-layer spectral budget n, the energy ratio delta, the scaling alpha, and per-task learning rates.

free parameters (4)
  • n = 750, 2250, 5000, 10000 depending on task
    Number of Hartley coefficients trained per layer. It directly sets the parameter count n times L and is tuned per model and dataset.
  • delta = 0.7
    Energy ratio splitting selection between top-energy (delta) and random (1-delta) frequencies; chosen from ablation in Fig. 4 (tested 0.5 to 0.9).
  • alpha = 16.0 in instruction tuning; 300 in E2E and vision
    Scaling factor applied to the inverse-DHT update; differs across experiments (Tables 10, 11, 12).
  • learning rate = 1E-2 to 1.2E-1 (GLUE), 3E-3 (instruction), 2E-1 to 3E-1 (vision)
    Tuned per task; appears in appendix Tables 9, 11, 12.
assumptions (5)
  • standard math The 2D discrete Hartley transform is self-inverse up to normalization, so the inverse DHT equals the forward DHT.
    Used in Eq. (5) and in the claim that DHT simplifies forward and backward transforms; Eq. (2) omits the normalization constant.
  • domain assumption Pretrained weight matrices are energy-compact in the Hartley domain, so top-energy coefficients are the most informative for fine-tuning.
    Central selection heuristic in Section 3.1, Eq. (3); Fig. 3 shows histograms but no quantitative validation of the assumption.
  • domain assumption A single selection budget n and energy ratio delta can be shared across all layers.
    Algorithm 1 applies consistent n and delta per layer; no per-layer importance measure is learned.
  • domain assumption GPT-4 scoring on MT-Bench and Vicuna is a reliable evaluation of fine-tuned instruction-following quality.
    Table 4's instruction-tuning comparisons rely entirely on GPT-4-generated scores.
  • domain assumption Published baseline numbers copied from prior work are accurate.
    Tables 3 and 6 mark some results as sourced from previous studies; the authors do not rerun these baselines.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation." pith.science (2026). https://pith.science/paper/LXV2JR7L

@misc{pith2026250205539,
  author       = {Pith},
  title        = {Pith review of: SSH: Sparse Spectrum Adaptation via Discrete Hartley Transformation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LXV2JR7L}},
  note         = {Machine review of arXiv:2502.05539}
}
read the original abstract

Low-rank adaptation (LoRA) has been demonstrated effective in reducing the trainable parameter number when fine-tuning a large foundation model (LLM). However, it still encounters computational and memory challenges when scaling to larger models or addressing more complex task adaptation. In this work, we introduce Sparse Spectrum Adaptation via Discrete Hartley Transformation (SSH), a novel approach that significantly reduces the number of trainable parameters while enhancing model performance. It selects the most informative spectral components across all layers, under the guidance of the initial weights after a discrete Hartley transformation (DHT). The lightweight inverse DHT then projects the spectrum back into the spatial domain for updates. Extensive experiments across both single-modality tasks such as language understanding and generation and multi-modality tasks such as video-text understanding demonstrate that SSH outperforms existing parameter-efficient fine-tuning (PEFT) methods while achieving substantial reductions in computational cost and memory requirements.

Figures

Figures reproduced from arXiv: 2502.05539 by the authors.

Figure 1
Figure 1. Performance and computation comparison of fine-tuning methods in NLP and CV Tasks. (a) For NLP on LLaMA3.1-8B, SSH achieves 7.93 GPT-4 score, closely matching full fine-tuning’s 7.95 score, while using less than 0.1% of the parameters. (b) In CV tasks, SSH achieves 77.4% accuracy, matching the performance of full fine-tuning with significantly fewer parameters. (c) & (d) SSH reduces up to 55% of GFLOPs compared to F… view at source ↗
Figure 2
Figure 2. Overview of Sparse Spectrum Adaptation via Discrete Hartley Transform (SSH). First, the discrete Hartley transform (DHT) is applied to the pretrained weights to extract and retain the most important frequency components. Then, a selective process identifies specific spectral coefficients to be learned as trainable parameters, which are organized into a spectral matrix. Finally, the modified spectral matrix is transf… view at source ↗
Figure 3
Figure 3. Visual representation of the RoBERTa attention mechanism’s key and value matrices before and after discrete Hartley transform (DHT). (a)(b) show the original weight dis￾tributions of the key and value matrices, respectively. (d)(e) depict the transformed DHT values, demonstrating effective spectral compression. Heatmaps (c)(f) illustrate the output weights before and after DHT, highlighting the achieved spar￾sity an… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Ablation study of SSH on GLUE tasks illustrat￾ing the effect of varying energy ratios (δ) on performance with RoBERTa-base (n=750). Performance is normalized to δ = 0.5, showing optimal balance and diversity in spectral representation at δ = 0.7. and ViT-L across four …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 24 canonical work pages

  1. [1]

    Ehsan Aghapour, Yixian Shen, Dolly Sapra, Andy Pimentel, and Anuj Pathania. 2024. Piqi: Partially quantized dnn inference on hmpsocs. In Proceedings of the 29th ACM/IEEE International Symposium on Low Power Electronics and Design, pages 1--6

    work page 2024

  2. [2]

    Seyedarmin Azizi, Souvik Kundu, and Massoud Pedram. 2024. Lamda: Large model fine-tuning via spectrally decomposed low-dimensional adaptation. arXiv preprint arXiv:2406.12832

    arXiv 2024

  3. [3]

    Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90\ See https://vicuna. lmsys. org (accessed 14 April 2023), 2(3):6

    2023

  4. [4]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and Andrea Vedaldi. 2014. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606--3613

    2014

  5. [5]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  6. [6]

    V \' tor A Coutinho, Renato J Cintra, and F \'a bio M Bayer. 2021. Low-complexity three-dimensional discrete hartley transform approximations for medical image compression. Computers in Biology and Medicine, 139:105018

    work page 2021

  7. [7]

    Tim Dettmers, Mike Lewis, Sam Shleifer, and Luke Zettlemoyer. 2022. https://arxiv.org/abs/2110.02861 8-bit optimizers via block-wise quantization . Preprint, arXiv:2110.02861

    arXiv 2022

  8. [8]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient finetuning of quantized llms. Advances in Neural Information Processing Systems, 36

    2024

Show all 60 references
  1. [9]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint...

    2020 arXiv

  2. [10]

    Ziqi Gao, Qichao Wang, Aochuan Chen, Zijing Liu, Bingzhe Wu, Liang Chen, and Jia Li. 2024. Parameter-efficient fine-tuning with discrete fourier transform. arXiv preprint arXiv:2405.03003

    2024 arXiv

  3. [11]

    Xiaotian Guo, Quan Jiang, Yixian Shen, Andy D Pimentel, and Todor Stefanov. 2024. Easter: Learning to split transformers at the edge robustly. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 43(11):3626--3637

    2024

  4. [12]

    Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. 2021. Towards a unified view of parameter-efficient transfer learning. arXiv preprint arXiv:2110.04366

    2021 arXiv

  5. [13]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217--2226

    2019

  6. [14]

    Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. Advances in neural information processing systems, 28

    2015

  7. [15]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790--2799. PMLR

    2019

  8. [16]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. Lora: Low-rank adaptation of large language models. International Conference on Learning Representations

    2022

  9. [17]

    Jia-Hong Huang, Yixian Shen, Hongyi Zhu, Stevan Rudinac, and Evangelos Kanoulas. 2024 a . Gradient weight-normalized low-rank projection for efficient llm training. arXiv preprint arXiv:2412.19616

    2024 arXiv

  10. [18]

    Jia-Hong Huang, Hongyi Zhu, Yixian Shen, Stevan Rudinac, and Evangelos Kanoulas. 2025. Image2text2image: A novel framework for label-free evaluation of image-to-text generation with text-to-image diffusion models. In International Conference on Multimedia Modeling, pages 413--...

    2025

  11. [19]

    Jia-Hong Huang, Hongyi Zhu, Yixian Shen, Stevan Rudinac, Alessio M Pacces, and Evangelos Kanoulas. 2024 b . A novel evaluation framework for image2text generation. arXiv preprint arXiv:2408.01723

    2024 arXiv

  12. [20]

    Raisa Islam and Owana Marzia Moushi. 2024. Gpt-4o: The cutting-edge advancement in multimodal llm. Authorea Preprints

    2024

  13. [21]

    Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki Markus Asano. 2023. Vera: Vector-based random matrix adaptation. arXiv preprint arXiv:2310.11454

    2023 arXiv

  14. [22]

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554--561

    2013

  15. [23]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. Technical Report TR-2009

    2009

  16. [24]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. https://arxiv.org/abs/1910.13461 Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehensi...

    2019 arXiv

  17. [25]

    Vladislav Lialin, Sherin Muckatira, Namrata Shivagunde, and Anna Rumshisky. 2023. Relora: High-rank training through low-rank updates. In Workshop on Advancing Neural Network Training: Computational Efficiency, Scalability, and Resource Optimization (WANT@ NeurIPS 2023)

    2023

  18. [26]

    Baohao Liao, Yan Meng, and Christof Monz. 2023 a . Parameter-efficient fine-tuning without introducing new latency. arXiv preprint arXiv:2305.16742

    2023 arXiv

  19. [27]

    Baohao Liao, Shaomu Tan, and Christof Monz. 2023 b . Make pre-trained model reversible: From parameter to memory efficient fine-tuning. Advances in Neural Information Processing Systems, 36

    2023

  20. [28]

    Zhaojiang Lin, Andrea Madotto, and Pascale Fung. 2020. Exploring versatile generative language model via parameter-efficient transfer learning. arXiv preprint arXiv:2004.03829

    2020 arXiv

  21. [29]

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024 a . Dora: Weight-decomposed low-rank adaptation. International Conference on Machine Learning

    2024

  22. [30]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692

    2019 arXiv

  23. [31]

    Zeyu Liu, Souvik Kundu, Anni Li, Junrui Wan, Lianghao Jiang, and Peter Anthony Beerel. 2024 b . Aflora: Adaptive freezing of low rank adaptation in parameter efficient fine-tuning of large models. arXiv preprint arXiv:2403.13269

    2024 arXiv

  24. [32]

    Mengchao Ma, Qianzhen Sun, Xicheng Gao, Guan Wang, Huaxia Deng, Yi Zhang, Qingtian Guan, and Xiang Zhong. 2021. High-efficiency single-pixel imaging using discrete hartley transform. AIP Advances, 11(7)

    2021

  25. [33]

    Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745

    2018 arXiv

  26. [34]

    Sobhan Niknam, Yixian Shen, Anuj Pathania, and Andy D Pimentel. 2023. 3d-ttp: Efficient transient temperature-aware power budgeting for 3d-stacked processor-memory systems. In 2023 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pages 1--6. IEEE

    2023

  27. [35]

    Jekaterina Novikova, Ond r ej Du s ek, and Verena Rieser. 2017. The e2e dataset: New challenges for end-to-end generation. arXiv preprint arXiv:1706.09254

    2017 arXiv

  28. [36]

    Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. 2012. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498--3505. IEEE

    2012

  29. [37]

    Jonas Pfeiffer, Aishwarya Kamath, Andreas R \"u ckl \'e , Kyunghyun Cho, and Iryna Gurevych. 2020. Adapterfusion: Non-destructive task composition for transfer learning. arXiv preprint arXiv:2005.00247

    2020 arXiv

  30. [38]

    William H Press. 2007. Numerical recipes 3rd edition: The art of scientific computing. Cambridge university press

    2007

  31. [39]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  32. [40]

    K Vijila Rani, M Eugine Prince, P Sujatha Therese, P Josephin Shermila, and E Anna Devi. 2024. Content-based medical image retrieval using fractional hartley transform with hybrid features. Multimedia Tools and Applications, 83(9):27217--27242

    2024

  33. [41]

    Adithya Renduchintala, Tugrul Konuk, and Oleksii Kuchaiev. 2024. https://arxiv.org/abs/2311.09578 Tied-lora: Enhancing parameter efficiency of lora with weight tying . Preprint, arXiv:2311.09578

    2024 arXiv

  34. [42]

    Andreas R \"u ckl \'e , Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych. 2020. Adapterdrop: On the efficiency of adapters in transformers. arXiv preprint arXiv:2010.11918

    2020 arXiv

  35. [43]

    Yanxin Shen, Lun Wang, Chuanqi Shi, Shaoshuai Du, Yiyi Tao, Yixian Shen, and Hang Zhang. 2024 a . Comparative analysis of listwise reranking with large language models in limited-resource language contexts. arXiv preprint arXiv:2412.20061

    2024 arXiv

  36. [44]

    Yixian Shen, Qi Bi, Jia-Hong Huang, Hongyi Zhu, and Anuj Pathania. 2024 b . Parameter-efficient fine-tuning via selective discrete cosine transform. arXiv preprint arXiv:2410.09103

    2024 arXiv

  37. [45]

    Yixian Shen, Sobhan Niknam, Anuj Pathania, and Andy D Pimentel. 2023 a . Thermal management for s-nuca many-cores via synchronous thread rotations. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1--6. IEEE

    2023

  38. [46]

    Yixian Shen, Leo Schreuders, Anuj Pathania, and Andy D Pimentel. 2023 b . Thermal management for 3d-stacked systems via unified core-memory power regulation. ACM Transactions on Embedded Computing Systems, 22(5s):1--26

    2023

  39. [47]

    Yixian Shen, Jun Xiao, and Andy D Pimentel. 2022. Tcps: a task and cache-aware partitioned scheduler for hard real-time multi-core systems. In Proceedings of the 23rd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems, pages 37--49

    2022

  40. [48]

    Yixian Shen, Hang Zhang, Yanxin Shen, Lun Wang, Chuanqi Shi, Shaoshuai Du, and Yiyi Tao. 2024 c . Altgen: Ai-driven alt text generation for enhancing epub accessibility. arXiv preprint arXiv:2501.00113

    2024 arXiv

  41. [49]

    Jianheng Tang, Jiajin Li, Ziqi Gao, and Jia Li. 2022. Rethinking graph neural networks for anomaly detection. In International Conference on Machine Learning, pages 21076--21089. PMLR

    2022

  42. [50]

    Yiyi Tao, Yixian Shen, Hang Zhang, Yanxin Shen, Lun Wang, Chuanqi Shi, and Shaoshuai Du. 2024. Robustness of large language models against adversarial attacks. arXiv preprint arXiv:2412.17011

    2024 arXiv

  43. [51]

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford alpaca: An instruction-following llama model

    2023

  44. [52]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  45. [53]

    Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. https://arxiv.org/abs/1804.07461 Glue: A multi-task benchmark and analysis platform for natural language understanding . Preprint, arXiv:1804.07461

    2019 arXiv

  46. [54]

    Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

    2022 arXiv

  47. [55]

    Kai Xu, Minghai Qin, Fei Sun, Yuhao Wang, Yen-Kuang Chen, and Fengbo Ren. 2020. Learning in the frequency domain. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1740--1749

    2020

  48. [56]

    Zai Yang and Lihua Xie. 2016. Exact joint sparse frequency recovery via optimization methods. IEEE Transactions on Signal Processing, 64(19):5145--5157

    2016

  49. [57]

    Q Zhang, M Chen, A Bukharin, P He, Y Cheng, W Chen, and T Zhao. 2023. Adaptive budget allocation for parameter-efficient fine-tuning. preprint (2023). arXiv preprint arXiv:2303.10512

    2023 arXiv

  50. [58]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36

    2024

  51. [59]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  52. [60]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.