Pith. sign in

REVIEW 5 major objections 3 minor 1 cited by

Dual Decomposition of Weights and Singular Value Low Rank Adaptation

T0 review · 5 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that initializing a low-rank adapter from the principal singular components of the pre-trained weight, while training magnitude and direction separately, consistently beats LoRA, DoRA, and PiSSA on reasoning and…

desk verdict A plausible and genuinely new DoRA+PiSSA hybrid whose headline configuration is contradicted by the paper's own variant experiments. read the letter →

arxiv 2505.14367 v2 pith:PEDJMA4K submitted 2025-05-20 cs.CL

Jialong Han , Si Zhang , Ke Zhang This is my paper
classification cs.CL
keywords parameter-efficientfine-tuninglow-rankadaptationsingularvaluedecompositionmagnitude-directionDoRAPiSSALLMoptimizationstability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes DuDe, a parameter-efficient fine-tuning method that splits each pre-trained weight matrix into a trainable magnitude vector and a direction matrix, then initializes the direction adapter from the matrix's top singular components rather than randomly. The authors claim this combination—DoRA's magnitude-direction decomposition plus PiSSA's SVD-based initialization—makes training more stable and transfers pre-trained knowledge better than LoRA, DoRA, or PiSSA. In their experiments DuDe wins on eight commonsense reasoning tasks across three models, on the graduate-level GPQA benchmark across four models, on MMLU at every rank from 2 to 32, and on GSM8K across five random seeds, with the largest gains on knowledge-intensive tasks. If the claim is right, a small, cheap initialization change to existing low-rank adapters buys both higher accuracy and lower variance across seeds.

What carries the argument

The load-bearing identity is the reparameterized weight $W' = m\,\frac{W_f + BA}{\|W_f + BA\|_c}$ with $W_f = W_0 - U_r\Sigma_r V_r^\top$ frozen and the low-rank factors initialized as $B = U_r\sqrt{\Sigma_r}$, $A = \sqrt{\Sigma_r}V_r^\top$, where $U_r\Sigma_rV_r^\top$ is the truncated SVD of the pre-trained weight $W_0$. The magnitude vector $m$ carries the scale of each output column during training, while the normalized direction part carries the geometry; the SVD initialization places the low-rank update inside the principal subspace of $W_0$, so the model begins at the pre-trained weights and the trainable directions are already aligned with the most informative singular directions. The paper's gradient analysis then shows that updates to $W_0$ are rescaled and projected onto the orthogonal complement of $W_0$, which is what it credits for the improved stability.

What would settle it

Compare DuDe against a variant that initializes the direction adapter with the bottom-$r$ singular components of $W_0$ (or random orthogonal directions) on a suite of tasks; if the bottom-r or random variant matches DuDe's accuracy, the claim that principal singular components are the informative subspace is falsified. Also, check the initialized forward pass: if $W' \neq W_0$ for the stated $m$ initialization, the equivalence claim in Eq. (7) is falsified.

Watch

Extended reading notes

Core claim

DuDe's central claim is that the two weaknesses of LoRA-style adapters—training instability and poor use of pre-trained knowledge—both stem from random initialization, and both can be fixed by decomposing each weight into magnitude and direction while seeding the direction adapter with the pre-trained weight's principal singular components. Concretely, the tuned weight is $W' = m\,\frac{W_f + BA}{\|W_f + BA\|_c}$, where $W_f = W_0 - U_r\Sigma_r V_r^\top$ is frozen, $B = U_r\sqrt{\Sigma_r}$ and $A = \sqrt{\Sigma_r}V_r^\top$ are the trainable low-rank factors, and $m$ is a trainable per-column magnitude. The gradient of the loss with respect to $W_0$ is scaled by $m/\|W_0\|_c$ and projected onto the orthogonal complement of $W_0$, which the paper argues aligns the gradient covariance closer to the identity and therefore stabilizes optimization; because $W_0 = W_f + \Delta W$, these benefits carry over to the update. Across seven models and four benchmarks, the paper reports that DuDe consistently outperforms LoRA, DoRA, and PiSSA, with up to 48.35% accuracy on MMLU, 62.53% (±1.59) on GSM8K, and large margins on GPQA such as 39.90% versus 30.81% for LoRA on Phi4 small.

Load-bearing premise

The load-bearing premise is that the top-$r$ singular directions of the pre-trained weight matrix form the best subspace from which to start the direction adapter; if that subspace is not the most informative for a downstream task, DuDe's advantage over random initialization would not hold, and the paper does not specify how the magnitude vector $m$ is initialized, so the claimed identity $W'=W_0$ at initialization is asserted rather than demonstrated.

Editorial extensions

If this is right

  • DuDe outperforms LoRA, DoRA, and PiSSA on all eight commonsense reasoning tasks averaged over Qwen1.5-7B, Qwen2.5-32B, and LLaMA2-13B, with the largest single-task gain being +11.76 points over LoRA on HellaSwag with LLaMA2-13B.
  • On GPQA, DuDe beats every baseline on all four models, with up to 9.09-point gains over LoRA (Phi4 small).
  • On MMLU with ranks 2 through 32, DuDe beats LoRA, DoRA, and PiSSA at every rank and every category, reaching 48.35% average accuracy at rank 32.
  • Across five random seeds on GSM8K, DuDe averages 62.53% with ±1.59 standard deviation, and its worst seed still beats the best seed of LoRA, DoRA, and PiSSA.
  • DuDe's training loss and gradient norm track full fine-tuning more closely than the baselines, which the paper interprets as evidence of more stable optimization.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: Section 4.6 suggests the split of singular values between the two factors matters—DuDe A (singular values on $A$) beats the symmetric split on GSM8K but not on MMLU—so the optimal split may depend on the task's spectral structure; a testable rule could be derived by measuring the spectral decay of fine-tuning updates.
  • Editorial extension: the gradient analysis implies DuDe's update direction is the full-fine-tuning gradient projected away from $W_0$; one could directly compare the cosine similarity of DuDe's and full fine-tuning's per-step updates and test whether that similarity predicts task accuracy.
  • Editorial extension: if the principal-subspace assumption is the mechanism, DuDe should help most when the downstream task aligns with the pre-trained representation's top singular directions; comparing DuDe against random-initialized DoRA on tasks with deliberately shifted input distributions would isolate the effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 3 minor

Summary. The paper proposes DuDe, a parameter-efficient fine-tuning method that combines DoRA's magnitude-direction decomposition with PiSSA's SVD-based initialization. The weight matrix is decomposed as W0 = Wf + BA, with A and B initialized from the top-r singular components, and the fine-tuned weight is written as W' = m (Wf + BA)/||Wf + BA||_c. The authors claim that this dual decomposition improves optimization stability and preserves pre-trained knowledge, and they support this with a gradient analysis and experiments on commonsense reasoning, GPQA, MMLU, and GSM8K across several model families.

Significance. If the claimed gains were robust, DuDe would be a useful contribution to the PEFT literature: it is a natural combination of two established ideas, and the experimental coverage is broad, spanning multiple model scales and task types. The paper also contains a candid limitations section and a single multi-seed robustness study, which are to its credit. However, the central claim of a principled and consistently superior initialization is currently undermined by the paper's own variant comparison in Section 4.6, by an incomplete theoretical derivation, and by the absence of uncertainty estimates for most of the headline tables. The contribution is therefore not yet established at the level claimed.

major comments (5)
  1. [Section 4.6 / Table 4] Table 4 shows that the two alternative singular-value splits, DuDe A and DuDe B, outperform the headline DuDe configuration on GSM8K (67.48% and 66.72% vs. 64.22%), and that DuDe A achieves a higher MMLU weighted average than DuDe (46.62% vs. 46.52%); the text itself states that DuDe A 'appears to be the most promising'. Because all three variants realize the same matrix W0 = Wf + BA at initialization, they share the gradient analysis of Section 3.3, and Eqs. (8)-(9) cannot distinguish between them. The paper's central claim that Eqs. (5)-(6) define a uniquely principled initialization is therefore contingent on an unexplained design choice that is actually contradicted by the reported evidence.
  2. [Section 3.3, Eq. (9)] Equation (9) is not well-defined as written: the norm ||W0||_c introduced in Eq. (2) is a column-wise norm and hence a vector, but the expression W0 W0^T / ||W0||_c^2 treats it as a scalar, or would require a row-wise normalization; no derivation or notational convention is supplied. Since this equation is the basis for the claimed projection onto the orthogonal complement of W0 and for the covariance-alignment argument, the theoretical analysis is currently incomplete.
  3. [Section 3.2, Eq. (7)] The assertion that the fine-tuned weight W' is equivalent to the original weight W0 at initialization is not demonstrated because the initialization of the magnitude vector m is never specified. At initialization Eq. (7) gives W' = m W0 / ||W0||_c, which equals W0 only if m is initialized to ||W0||_c (or an equivalent re-scaling is applied). The paper's claim that DuDe preserves pre-trained behavior depends on this unstated choice.
  4. [Section 3.3] The statement that 'the gradient ∂L/∂W0 is equivalent to ∂L/∂∆W' is followed by the claim that this gradient contains more stable and informative signals than LoRA's gradient, but no explicit comparison with the LoRA gradient is derived. Moreover, because Eq. (9) depends only on W0, it is identical for DuDe, DuDe A, and DuDe B; the analysis therefore cannot explain the large performance differences reported in Table 4, and the theoretical argument does not support the specific choice of Eq. (5)-(6).
  5. [Sections 4.1-4.4, Tables 1-3] Most of the headline performance comparisons are reported as single runs without error bars or significance tests, and several advantages are very small (for example, Qwen2.5-32B average 84.89 vs. LoRA 84.86; MMLU r=2 weighted average 45.56 vs. PiSSA 45.53). The claim that DuDe 'consistently outperforms' the baselines is therefore not established for these cases. The five-seed study in Section 4.5 is a good step, but it covers only one model and one task.
minor comments (3)
  1. [Figure 1 caption] The caption states that LoRA initializes A in R^{r x d}, while the text and Eq. (5) use A in R^{r x k}; these dimensions should be harmonized.
  2. [Table 4] The column header 'DuDeB' is missing a space; it should be 'DuDe B' for consistency with 'DuDe A'.
  3. [Reproducibility] No code or configuration files are provided; for a method whose contribution is an initialization procedure, this limits the reproducibility of the reported experiments and of the variant comparison in Section 4.6.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: DuDe is an empirical composition of DoRA and PiSSA evaluated on external benchmarks, with no fitted parameter renamed as a prediction.

full rationale

The paper's central claim is that DuDe, which combines DoRA's magnitude-direction decomposition with PiSSA's SVD-based initialization, improves PEFT performance and stability. The reported gains come from direct evaluation on external benchmarks (commonsense reasoning, GPQA, MMLU, GSM8K), not from a derivation that assumes the conclusion. The gradient analysis in Section 3.3 derives formulas from the definition of W' and is post hoc rather than circular: it explains why the decomposition might help, but the performance claims do not reduce to the formulas. Section 4.6 shows that alternative singular-value splits (DuDe A and DuDe B) can outperform the headline configuration, which is a legitimate design-choice concern, but it is not a circularity: no parameter is fitted to the test set and then called a prediction, and no equation forces the headline choice. The claim that W' equals W0 at initialization is under-specified because the initialization of m is not stated, but this is an omission or correctness issue, not a self-referential reduction. The method builds on DoRA and PiSSA via external citations; there are no load-bearing self-citations, and no uniqueness theorem is imported from the authors' own prior work. Accordingly, the derivation chain is self-contained with respect to circularity, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

DuDe introduces no new physical or mathematical entities; it reuses the magnitude-direction decomposition from DoRA and the SVD initialization from PiSSA. The ledger captures the load-bearing assumptions imported from prior work and the unstated initialization of the magnitude vector, which the central claim depends on.

free parameters (2)
  • rank r = 16 (main), 2-32 (sweep)
    The adapter rank is chosen by hand and is central to the method's capacity. DuDe's reported advantage over LoRA grows with rank (largest at r=32), so the comparison outcome depends on this choice.
  • magnitude vector m initialization = not reported
    Eq. (7) introduces a trainable magnitude vector m, but the paper never states its initial value. DoRA initializes m to the column-wise norms of W0; DuDe presumably does the same, but the reader must assume this, and the claimed equivalence W' = W0 at initialization depends on it.
assumptions (4)
  • domain assumption Fine-tuning updates have low intrinsic rank (Aghajanyan et al., 2021)
    Section 3.1 builds on the LoRA premise that updates can be captured in a low-rank subspace; DuDe inherits this assumption without new justification.
  • domain assumption Magnitude-direction decomposition preserves the pre-trained function at initialization
    Section 3.2 Eq. (7) requires W' = W0 when BA = 0 and m is set appropriately, but the initialization of m is not specified, so this foundational property is asserted rather than shown.
  • domain assumption Top-r singular components of W0 form the most informative subspace for adaptation
    Eqs. (3)-(6) initialize the adapter from the top-r SVD components following PiSSA; the paper does not provide independent evidence that this subspace is optimal for downstream tasks in the magnitude-direction framework.
  • standard math Weight-normalization gradient projection formula (Salimans and Kingma, 2016)
    Eq. (9) uses the projection (I - W0 W0^T / ||W0||_c^2) with column-wise norms, but the notation is ambiguous because ||W0||_c is a vector, not a scalar; the derivation is not reproduced and the formula may be ill-defined.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dual Decomposition of Weights and Singular Value Low Rank Adaptation." pith.science (2026). https://pith.science/paper/PEDJMA4K

@misc{pith2026250514367,
  author       = {Pith},
  title        = {Pith review of: Dual Decomposition of Weights and Singular Value Low Rank Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PEDJMA4K}},
  note         = {Machine review of arXiv:2505.14367}
}
abstract

Parameter-Efficient Fine-Tuning (PEFT) has emerged as a critical paradigm for adapting Large Language Models (LLMs) to downstream tasks, among which Low-rank Adaptation (LoRA) represents one of the most widely adopted methodologies. However, existing LoRA-based approaches exhibit two fundamental limitations: unstable training dynamics and inefficient knowledge transfer from pre-trained models, both stemming from random initialization of adapter parameters. To overcome these challenges, we propose DuDe, a novel approach that decomposes weight matrices into magnitude and direction components, employing Singular Value Decomposition (SVD) for principled initialization. Our comprehensive evaluation demonstrates DuDe's superior performance and robustness, achieving up to 48.35\% accuracy on MMLU and 62.53\% ($\pm$ 1.59) accuracy on GSM8K. Our theoretical analysis and empirical validation collectively demonstrate that DuDe's decomposition strategy enhances optimization stability and better preserves pre-trained representations, particularly for domain-specific tasks requiring specialized knowledge. The combination of robust empirical performance and rigorous theoretical foundations establishes DuDe as a significant contribution to PEFT methodologies for LLMs.

Figures

Figures reproduced from arXiv: 2505.14367 by the authors.

Figure 1
Figure 1. The blue parts in the figure represent frozen components, while the orange parts represent trainable [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Comparison of Full finetuning, DuDe and other PEFT methods on Mistral 7B v0.2 model: (a) Training [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Average accuracy of DuDe and LoRA on MMLU tasks with different seeds. The detailed performance trajectory across dif￾ferent seeds is visualized in [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: Performance comparison between LoRA and DuDe on MMLU tasks with varying rank settings. (a) Average accuracy across all MMLU categories shows DuDe consistently outperforming LoRA, especially at larger ranks. (b) Weighted average accuracy demon￾strates similar trends, wi…

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. A Six-Dimensional Taxonomy of Post-Training Adaptation Techniques with Applications in AI Governance

    cs.LG 2026-08 conditional novelty 6.0 of 10

    A new taxonomy characterizes 48 post-training AI adaptation techniques on six axes and maps them to regulatory documentation requirements.

Reference graph

Works this paper leans on

39 extracted references · 11 canonical work pages · cited by 1 Pith paper

  1. [1]

    Hewett, Mojan Javaheripi, and Piero Kauffmann

    Marah Abdin, Jyoti Aneja, Harkirat Behl, S \'e bastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, and Piero Kauffmann. 2024. https://doi.org/10.48550/arXiv.2412.08905 Phi-4 Technical Report . arXiv preprint arXiv:2412.08905

    arXiv doi:10.48550/arxiv.2412.08905 2024

  2. [2]

    Armen Aghajanyan, Sonal Gupta, and Luke Zettlemoyer. 2021. https://doi.org/10.18653/v1/2021.acl-long.568 Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning . In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ( ...

    doi:10.18653/v1/2021.acl-long.568 2021

  3. [3]

    Klaudia Ba azy, Mohammadreza Banaei, Karl Aberer, and Jacek Tabor. 2024. https://doi.org/10.48550/arXiv.2405.17604 LoRA-XS : Low-Rank Adaptation with Extremely Small Number of Parameters . arXiv preprint arXiv:2405.17604

    arXiv doi:10.48550/arxiv.2405.17604 2024

  4. [4]

    Elad Ben Zaken, Yoav Goldberg, and Shauli Ravfogel. 2022. https://doi.org/10.18653/v1/2022.acl-short.1 B it F it: Simple parameter-efficient fine-tuning for transformer-based masked language-models . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 1--9, Dublin, Ireland. Association...

    doi:10.18653/v1/2022.acl-short.1 2022

  5. [5]

    Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2019. https://doi.org/10.48550/arXiv.1911.11641 PIQA : Reasoning about Physical Commonsense in Natural Language . arXiv preprint arXiv:1911.11641

    arXiv doi:10.48550/arxiv.1911.11641 2019

  6. [6]

    u y \"u kaky \

    Kerim B \"u y \"u kaky \"u z. 2024. https://doi.org/10.48550/arXiv.2406.01775 OLoRA : Orthonormal Low-Rank Adaptation of Large Language Models . arXiv preprint arXiv:2406.01775

    arXiv doi:10.48550/arxiv.2406.01775 2024

  7. [7]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 BoolQ : Exploring the Surprising Difficulty of Natural Yes / No Questions . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics : Human Language...

    doi:10.18653/v1/n19-1300 2019

  8. [8]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. https://doi.org/10.48550/arXiv.1803.05457 Think you have Solved Question Answering ? Try ARC , the AI2 Reasoning Challenge . arXiv preprint arXiv:1803.05457

    arXiv doi:10.48550/arxiv.1803.05457 2018

Show all 39 references
  1. [9]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . arXiv preprint arXiv:2110.14168

    2021 arXiv

  2. [10]

    OpenCompass Contributors. 2023. Open Compass : A Universal Evaluation Platform for Foundation Models . https://github.com/open-compass/opencompass

    2023

  3. [11]

    Clark, and Mehdi Rezagholizadeh

    Ali Edalati, Marzieh Tahaei, Ivan Kobyzev, Vahid Partovi Nia, James J. Clark, and Mehdi Rezagholizadeh. 2022. https://doi.org/10.48550/arXiv.2212.10650 KronA : Parameter Efficient Tuning with Kronecker Adapter . arXiv preprint arXiv:2212.10650

    2022 doi arXiv

  4. [12]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. https://doi.org/10.1109/ICCV.2015.123 Delving Deep into Rectifiers : Surpassing Human-Level Performance on Imagenet Classification . In 2015 IEEE International Conference on Computer Vision ( ICCV ) , pages 1026--103...

    2015 doi

  5. [13]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/pdf?id=7Bywt2mQsCe Measuring Mathematical Problem Solving With the MATH Dataset . In Thirty-Fifth Conference on Neural Information ...

    2021

  6. [14]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. https://arxiv.org/pdf/1902.00751 Parameter- Efficient Transfer Learning for NLP . In Proceedings of the 36th International Conf...

    2019 arXiv

  7. [15]

    Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu , Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/pdf?id=nZeVKeeFYf9 LoRA : Low-rank Adaptation of Large Language Models . In International Conference on Learning Representations

    2022

  8. [16]

    Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, and Lucile Saulnier

    Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, and Lucile Saulnier. 2023. https://doi.org/10.48550/arXiv.2310.06825 Mistral 7B . arXiv preprint arXiv:2310.06825

    2023 doi arXiv

  9. [17]

    Dawid J Kopiczko, Tijmen Blankevoort, and Yuki M Asano. 2024. https://openreview.net/pdf?id=NjNfLdxr3A VeRA : Vector-Based Random Matrix Adaptation . In International Conference on Learning Representations

    2024

  10. [18]

    Tao Lei. 2023. https://openreview.net/pdf?id=IyYyKov0Aj Conditional Adapters : Parameter-efficient Transfer Learning with Fast Inference . In Advances in Neural Information Processing Systems , volume 36, pages 8152--8172. Curran Associates, Inc

    2023

  11. [19]

    Martin, Kenneth L

    Yingzhou Li, Haizhao Yang, Eileen R. Martin, Kenneth L. Ho, and Lexing Ying. 2024. https://openreview.net/pdf?id=7NzgkEdGyr Parameter- Efficient Orthogonal Finetuning Via Butterfly Factorization . In The Twelfth International Conference on Learning Representations

    2024

  12. [20]

    Zhuang Li, Yuyang Chai, Terry Yue Zhuo, Lizhen Qu, Gholamreza Haffari, Fei Li, Donghong Ji, and Quan Hung Tran. 2023. https://doi.org/10.18653/v1/2023.findings-acl.398 FACTUAL : A benchmark for faithful and consistent textual scene graph parsing . In Findings of the Associatio...

    2023 doi

  13. [21]

    Baohao Liao, Yan Meng, and Christof Monz. 2023. https://doi.org/10.18653/v1/2023.acl-long.233 Parameter-efficient fine-tuning without introducing new latency . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    2023 doi

  14. [22]

    Vijay Lingam, Atula Tejaswi, Aditya Vavre, Aneesh Shetty, Gautham Krishna Gudur, Joydeep Ghosh, Alex Dimakis, Eunsol Choi, Aleksandar Bojchevski, and Sujay Sanghavi. 2024. https://doi.org/10.48550/arXiv.2405.19597 SVFT : Parameter-Efficient Fine-Tuning with Singular Vectors . ...

    2024 doi arXiv

  15. [23]

    Fangyu Liu, Guy Emerson, and Nigel Collier. 2023. https://doi.org/10.1162/tacl_a_00566 Visual spatial reasoning . Transactions of the Association for Computational Linguistics, 11:635--651

    2023 doi

  16. [24]

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. https://arxiv.org/pdf/2402.09353 DoRA : Weight-Decomposed Low-Rank Adaptation . In Forty-First International Conference on Machine Learning , pages 32100...

    2024 arXiv

  17. [25]

    Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. https://arxiv.org/pdf/2404.02948 PiSSA : Principal Singular Values and Singular Vectors Adaptation of Large Language Models . In Advances in Neural Information Processing Systems , volume 37, pages 121038--121072. Curran Associates, Inc

    2024 arXiv

  18. [26]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. https://doi.org/10.18653/v1/D18-1260 Can a Suit of Armor Conduct Electricity ? A New Dataset for Open Book Question Answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language ...

    2018 doi

  19. [27]

    Libo Qin, Qiguang Chen, Fuxuan Wei, Shijue Huang, and Wanxiang Che. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.163 Cross-lingual prompting: Improving zero-shot chain-of-thought reasoning across languages . In Proceedings of the 2023 Conference on Empirical Methods in Na...

    2023 doi

  20. [28]

    Qwen, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, and Fei Huang. 2025. https://doi.org/10.48550/arXiv.2412.15115 Qwen2.5 Technical Report . arXiv preprint arXiv:2412.15115

    2025 doi arXiv

  21. [29]

    David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. https://arxiv.org/pdf/2311.12022 GPQA : A Graduate-Level Google-Proof Q & A Benchmark . In First Conference on Language Modeling

    2024 arXiv

  22. [30]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. https://doi.org/10.1145/3474381 WinoGrande : An adversarial winograd schema challenge at scale . Communications of the ACM, 64(9):99--106

    2021 doi

  23. [31]

    Tim Salimans and Durk P Kingma. 2016. https://proceedings.neurips.cc/paper_files/paper/2016/hash/ed265bc903a5a097f61d3ec064d96d2e-Abstract.html Weight Normalization : A Simple Reparameterization to Accelerate Training of Deep Neural Networks . In Advances in Neural Information...

    2016

  24. [32]

    Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. https://doi.org/10.18653/v1/D19-1454 Social IQ a: Commonsense reasoning about social interactions . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9...

    2019 doi

  25. [33]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://doi.org/10.18653/v1/N19-1421 C ommonsense QA : A question answering challenge targeting commonsense knowledge . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associa...

    2019 doi

  26. [34]

    Qwen Team. 2024. https://qwenlm.github.io/blog/qwen1.5/ Introducing Qwen1.5

    2024

  27. [35]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, and Shruti Bhosale. 2023. https://doi.org/10.48550/arXiv.2307.09288 Llama 2: Open Foundation and Fine-Tuned Chat Models . arXiv preprint ...

    2023 doi arXiv

  28. [36]

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. https://openreview.net/pdf?id=N8N0hgNDRt Metamath: Bootstrap Your Own Mathematical Questions for Large Language Models . In The Twelfth Inte...

    2024

  29. [37]

    Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. https://doi.org/10.18653/v1/P19-1472 HellaSwag : Can a Machine Really Finish Your Sentence ? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 4791--4...

    2019 doi

  30. [38]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.