Pith. sign in

REVIEW 3 major objections 4 minor 54 references

Language Fusion for Parameter-Efficient Cross-lingual Transfer

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fusing English and target-language representations inside LoRA adapter bottlenecks improves cross-lingual downstream performance, with the largest gain on question answering.

desk verdict Solid new fusion mechanism with a confounded headline comparison; worth review, but the LoRA gains partly reflect extra source data. read the letter →

arxiv 2501.06892 v2 pith:ZLQ3VLZS submitted 2025-01-12 cs.CL

Philipp Borchert , Ivan Vulić , Marie-Francine Moens , Jochen De Weerdt This is my paper · ORCID
classification cs.CL
keywords cross-lingualtransferparameter-efficientfine-tuningLoRAadapterfusionrepresentationlow-resourcelanguagesmultilingualquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that cross-lingual transfer can be done by merging English and target-language representations inside the low-rank bottleneck of LoRA adapters, rather than by mixing text at the input level or adding new model parameters. It argues that this representation-level fusion lets task knowledge flow from a high-resource language into a target language during fine-tuning, and that the mechanism works across encoder-only, encoder-decoder, and decoder-only multilingual models. The payoff is parameter-efficient transfer: FLARE adds no parameters beyond standard LoRA, avoids the doubled sequence lengths of input-level fusion, and still beats LoRA fine-tuning, X-Mixup, and input-level fusion on natural language inference, sentiment analysis, and question answering. A sympathetic reader would take the central claim to be that the adapter bottleneck is not just a compression device but a place where two languages' representation spaces can be productively combined.

What carries the argument

The central object is the fusion adapter: a LoRA adapter whose down-projection maps both source and target hidden states into a shared low-dimensional bottleneck, where they are combined token-wise by a fusion function, after which the shared up-projection returns the result to full model dimension and adds it to the frozen attention output. Formally, the fused representation is $h=\varphi(v^S_{i+1}W_{\text{down}},\; v^T_i W_{\text{down}})$, with $v^S$ taken from the following transformer block to capture task-specific source information, and the final output is $v^T_{i+1}=hW_{\text{up}}+v_0$. The fusion functions examined are element-wise addition, element-wise multiplication, a cross-attention variant, and their ReLU-gated versions, with addition plus ReLU giving the best average results. This mechanism does the work of transferring knowledge because it keeps the two languages in separate computation streams until the moment of fusion, then forces their aligned token representations to interact in the compressed rank space that the task head actually reads from.

What would settle it

Permute the source-language token order before fusing with the target representation inside the adapter bottleneck and measure downstream accuracy: if the shuffled-source result stays close to aligned FLARE, then token-level alignment is not carrying the transfer and the method's stated mechanism is not responsible for its gains.

Watch

Extended reading notes

Core claim

The paper introduces FLARE, which runs the source-language input through the frozen, adapter-free base model and the target-language input through the same model with LoRA adapters, extracts layer-wise hidden states from each, and fuses them inside the adapter bottleneck through a lightweight function such as element-wise addition or multiplication, optionally followed by ReLU. The fused low-rank representation is then up-projected and added to the frozen attention outputs. This token-wise fusion is applied in every transformer block, so the model continuously mixes source and target information during task adaptation instead of only in one layer or only at the input. Across XNLI, NusaX, and TyDiQA with XLM-R, mT5, Llama 3.1, and Gemma 2, FLARE outperforms standard LoRA fine-tuning on all three tasks, with the largest gains on question answering: +4.9 exact-match points for Llama 3.1 and +2.2 for Gemma 2 on TyDiQA. A variant called FLARE MT replaces the source-language forward pass with a latent translation from an MT encoder, cutting compute further at some performance cost for decoder-only models.

Load-bearing premise

For the fusion to transfer meaningful information, the down-projected source and target representations must be aligned token by token after machine translation, so that adding or multiplying them at each position pairs up corresponding meanings.

Editorial extensions

If this is right

  • FLARE's biggest wins are on generative question answering with decoder-only models, raising exact match by 4.9 points on Llama 3.1 and 2.2 points on Gemma 2 over standard LoRA fine-tuning.
  • The method is architecture-agnostic: it improves translate-train performance on XNLI and NusaX for encoder-only XLM-R, encoder-decoder mT5, and decoder-only Llama and Gemma, so the fusion idea transfers across model families.
  • Replacing the source-language forward pass with an MT encoder's latent translation gives a cheaper variant that still beats LoRA on encoder-style models, meaning cross-lingual transfer can be done without decoding a full source sentence.
  • FLARE remains competitive when the machine translation quality is lower (using NLLB 600M instead of 3.3B), suggesting that the fusion bottleneck tolerates noisier parallel data than the comparison methods.
  • Fusing only during training collapses NusaX performance by 30%, so the source-language representations are needed at inference time; the transfer is instance-dependent, not a static pattern learned from English data alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If token-wise alignment is genuinely the mechanism, then a direct test is to shuffle or shift the source token order before fusion; a large performance drop would confirm that positional matching carries the effect, while a small drop would suggest the method mainly injects extra features.
  • FLARE is explicitly agnostic to the source language, but the paper only uses English; the same adapter fusion could plausibly transfer from another high-resource language, such as Indonesian for NusaX languages, to related low-resource languages, which the paper itself lists as untested.
  • The fact that small bottlenecks (rank 8) work best on XLM-R hints that the cross-lingual signal is low-dimensional, which would imply that even cheaper adapters or other PEFT formulations could host the same fusion step.
  • Since FLARE tolerates lower-quality machine translations, a practical extension is to use it as a training recipe that lets smaller MT models suffice for building task data in low-resource languages, thereby lowering the compute budget for rapid-language adaptation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes FLARE, a parameter-efficient method for cross-lingual transfer that fuses English (source) and target-language representations inside LoRA adapter bottlenecks. During fine-tuning, source representations are extracted from the frozen base model at each transformer block, down-projected, combined with target representations via lightweight functions such as addition, multiplication, ReLU, or cross-attention, and then up-projected and added to attention outputs. The method is evaluated on XNLI, TyDiQA, and NusaX using XLM-R Large, mT5-XL, Llama 3.1 8B, and Gemma 2 9B, and compared against zero-shot cross-lingual transfer, translate-test, translate-train with standard LoRA, X-Mixup, and input-level fusion. The headline results are improvements on TyDiQA exact match over standard LoRA, e.g., +4.9% for Llama 3.1 and +2.2% for Gemma 2, with no added parameters in the default linear-fusion configuration. The paper also introduces FLARE MT, which fuses latent translations from an MT encoder into the adapters.

Significance. If the reported gains are reliable, FLARE is a potentially valuable contribution to parameter-efficient cross-lingual transfer: it keeps the frozen backbone intact, adds no parameters beyond standard LoRA in its default variant, and works across encoder-only, encoder-decoder, and decoder-only architectures. The experimental campaign is unusually broad for this area, with four model families, three tasks, five seeds, permutation tests, and a honest train-only ablation (Appendix E) that shows source-language representations must be present at inference for FLARE to work. However, the central causal claim—that fusing source and target representations inside the adapter rather than simply having access to source-language data drives the gains—is not yet established because the principal comparison is confounded and several headline differences are not statistically significant. The significance of the paper therefore depends on whether the authors can close this gap with additional controls and more careful claims.

major comments (3)
  1. [Section 3.3 and Section 4.1] The comparison between FLARE and 'standard LoRA fine-tuning' is confounded by source-language data availability. FLARE's forward pass processes the English source sentence xS at every training step and fuses its hidden states into the target adapter at every layer, whereas the LoRA baseline in the translate-train setting is trained only on translated target-language data. The gains in Table 1 (e.g., +4.9 exact match for Llama 3.1 on TyDiQA) may therefore reflect the extra information provided by seeing the English source, not the fusion mechanism itself. The input-level fusion baseline does use source and target data, but it concatenates the sequences and doubles the input length, which is a different computational and inductive setup. To isolate the fusion mechanism, please add a control in which the model also has access to the English source at training time but does not fuse layer-wise in the adapters—for example, pass the source through a frozen encoder and inject its pooled or projected representations only at the task head, while otherwise training a standard LoRA adapter on the target data. If such a control matches FLARE's performance, the central claim in the abstract would need to be reframed.
  2. [Section 5 and Table 9] The statement that 'FLARE consistently surpasses all baselines across various tasks' is not supported by the reported permutation tests. Table 9 shows many non-significant differences: FLARE vs. LoRA is non-significant on XLM-R TyDiQA (p=0.296), mT5-XL TyDiQA (p=0.181), Llama XNLI (p=0.321), and Gemma XNLI (p=0.221); FLARE vs. input-level fusion is non-significant on Llama TyDiQA (p=0.378), Gemma TyDiQA (p=0.505), and XLM-R TyDiQA (p=0.531). The only consistently significant gains over LoRA are on TyDiQA for the decoder-only models. Please report the significance status for every headline average in Table 1, or qualify the 'consistently surpasses' claim to reflect the actual support. The abstract's unqualified framing would otherwise overstate the evidence.
  3. [Section 3.2 and Figure 7] The token-wise fusion functions are defined positionally: S and T are down-projected source and target representations at the same token position, and the method's effectiveness is interpreted as evidence of positional alignment (Figures 5 and 7, and the discussion of ReLU as a way to 'restrict the propagation of misaligned information'). The paper does not, however, measure alignment quality or test how sensitive the result is to misalignment. Please quantify this assumption—for example, by comparing FLARE with source tokens shuffled against their aligned order, or by measuring token-level correspondence with MT alignment tools. This would also clarify why the gold-translation results in Table 4 show FLARE substantially underperforming input-level fusion on NusaX despite having perfect English input.
minor comments (4)
  1. [Section 5, 'On Latent MT Fusion'] This subsection refers to 'Table 2' for latent MT fusion results, but Table 2 reports the fusion-function ablation; the FLARE MT results are given in Table 1 and the significance tests in Table 9. Please correct the cross-reference.
  2. [Abstract and Section 3.1] The abstract says FLARE combines representations 'using lightweight linear transformations', but Section 3.2 also evaluates a cross-attention fusion function that introduces additional parameters. Please clarify that the parameter-free claim applies to the default linear/reLU variants, not to all evaluated fusion functions.
  3. [Table 9 title] The table reports p-values from Pitman permutation tests, but the caption does not state whether the tests are two-sided or one-sided, nor how the 90/95/99% thresholds correspond to *, **, ***. Please add these details.
  4. [Appendix E] The train-only ablation is valuable and honest, but the sentence 'the performance of the train-only FLARE variant decreased by 30%' should state whether 30% is a relative or absolute decrease and on which average across languages this is computed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: FLARE is an empirical method paper whose claims are validated on external benchmarks with held-out test splits.

full rationale

FLARE's central claim is that fusing English and target-language representations inside LoRA bottlenecks improves cross-lingual downstream performance. The paper supports this claim with external evidence: TyDiQA, XNLI, and NusaX results compared against zero-shot, translate-test, translate-train LoRA, X-Mixup, and input-level fusion baselines, averaged over five seeds and supplemented with Pitman permutation tests. No parameter is fitted to test-set outcomes, and no reported 'prediction' is derived from a definition that presupposes the result. Fusion functions, bottleneck sizes, and checkpoints are selected on validation data and then evaluated on held-out test sets, which is standard practice rather than circular fitting. The main comparison against standard LoRA is not perfectly controlled, because FLARE also receives English source representations during training, and some significance tests are non-significant; however, a confounded or underpowered experimental comparison is a validity concern, not a circular derivation. The self-citations present, such as Qu et al. for ReLU-based fusion and Schmidt et al. for a possible MLP projection extension, are background or forward-looking and are not load-bearing premises of the argument. The Limitations section and Appendix E openly state constraints and even show that restricting fusion to training only degrades NusaX performance by 30%, which undercuts rather than presupposes the method's claims. No load-bearing step in the derivation chain reduces by construction to its own inputs, so the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or conceptual entities; it introduces a new module composition. The free parameters are standard PEFT hyperparameters and fusion function choices, evaluated empirically.

free parameters (4)
  • fusion function = add+relu (main), add, mul, cross-attention evaluated
    The paper states the optimal fusion function is task-dependent and selected on validation data; Tables 2 and 3 compare options.
  • LoRA rank r = 64 (main); 8, 128 in Table 3
    Rank of the adapter bottleneck; the paper reports performance varies with r and uses r=64 as default.
  • LoRA scaling alpha = 128
    LoRA scaling factor used for all main experiments.
  • MT model choice = NLLB 3.3B for main results; NLLB 600M for Figure 4 and FLARE MT
    Choice of machine translation system affects the parallel data quality and is varied in experiments.
assumptions (4)
  • domain assumption Parallel corpora P = {xS, xT} are available during fine-tuning (typically via machine translation)
    Stated in Section 3.1; the method cannot run without source-target parallel data.
  • ad hoc to paper Token-wise fused representations in the adapter bottleneck are a meaningful and sufficient channel for cross-lingual transfer
    The central design assumption in Sections 3.1 and 3.2; no theoretical guarantee is provided.
  • domain assumption The base model fine-tuned on English task data provides source representations that retain task-relevant information
    Required by the transfer setup in Section 3.3.
  • ad hoc to paper Machine translations preserve enough token-level alignment for fusion
    The method relies on positional alignment; the paper does not measure alignment.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language Fusion for Parameter-Efficient Cross-lingual Transfer." pith.science (2026). https://pith.science/paper/ZLQ3VLZS

@misc{pith2026250106892,
  author       = {Pith},
  title        = {Pith review of: Language Fusion for Parameter-Efficient Cross-lingual Transfer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZLQ3VLZS}},
  note         = {Machine review of arXiv:2501.06892}
}
read the original abstract

Limited availability of multilingual text corpora for training language models often leads to poor performance on downstream tasks due to undertrained representation spaces for languages other than English. This 'under-representation' has motivated recent cross-lingual transfer methods to leverage the English representation space by e.g. mixing English and 'non-English' tokens at the input level or extending model parameters to accommodate new languages. However, these approaches often come at the cost of increased computational complexity. We propose Fusion forLanguage Representations (FLARE) in adapters, a novel method that enhances representation quality and downstream performance for languages other than English while maintaining parameter efficiency. FLARE integrates source and target language representations within low-rank (LoRA) adapters using lightweight linear transformations, maintaining parameter efficiency while improving transfer performance. A series of experiments across representative cross-lingual natural language understanding tasks, including natural language inference, question-answering and sentiment analysis, demonstrate FLARE's effectiveness. FLARE achieves performance improvements of 4.9% for Llama 3.1 and 2.2% for Gemma~2 compared to standard LoRA fine-tuning on question-answering tasks, as measured by the exact match metric.

Figures

Figures reproduced from arXiv: 2501.06892 by the authors.

Figure 1
Figure 1. Fusion of source and target representations in LoRA adapters inserted within the query and value matrices. The representations are fused in the adapter bottlenecks and the outputs are added + to the query and value outputs before softmax ⊗ activation. (Houlsby et al., 2019; Hu et al., 2022). This directly aligns with the XLT objectives, providing resource￾efficient language and task adaptation capabilities. In XLT, … view at source ↗
Figure 2
Figure 2. During the forward pass with FLARE, source language representations x S are processed by trans￾former block i and before fusion with target language representations x T . Source representations are obtained by inferencing the mPLM without the fusion adapters. adaptation in the target language by directing the model’s attention to task-relevant information. Thereby, the adapter bottleneck is used for cross￾lingual kn… view at source ↗
Figure 3
Figure 3. Illustration of the FLARE MT variant where projected encoder representations from an MT model are directly fused with target language representations within the fusion adapters in the mPLM. Encoder rep￾resentations from the MT model serve as latent transla￾tions, avoiding discretization in the decoder. that may be misaligned at the token level. By intro￾ducing non-linear transformation functions, we can restrict the… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Average performance differences on NusaX [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Average activation values for English and Acehnese in the first bottleneck query layer in XLM-R Large for the NusaX test set; add+relu fusion. 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Activation Value 0.00 0.25 0.50 0.75 1.00 Density English (en) Acehnese (ace) [PITH_FULL_IMAGE:fi…
Figure 6
Figure 6. Figure 6: Average activations in the adapters across all [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Activation values for individual instances [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 15 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Hizkiel Mitiku Alemayehu, Hamada M Zahera, and Axel-Cyrille Ngonga Ngomo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.1102 Error analysis of multilingual language models in machine translation: A case study of E nglish- A mharic translation . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 19758--1976...

    work page doi:10.18653/v1/2024.emnlp-main.1102 2024

  4. [4]

    Alan Ansell, Marinela Parovi \'c , Ivan Vuli \'c , Anna Korhonen, and Edoardo Ponti. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.242 Unifying cross-lingual transfer across scenarios of resource scarcity . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3980--3995, Singapore. Association for Computatio...

    doi:10.18653/v1/2023.emnlp-main.242 2023

  5. [5]

    Alan Ansell, Edoardo Ponti, Anna Korhonen, and Ivan Vuli \'c . 2022. https://doi.org/10.18653/v1/2022.acl-long.125 Composable sparse fine-tuning for cross-lingual transfer . In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1778--1796, Dublin, Ireland. Association for Computational Li...

    doi:10.18653/v1/2022.acl-long.125 2022

  6. [6]

    Mikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan, and Luke Zettlemoyer. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.399 Revisiting machine translation for cross-lingual classification . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6489--6499, Singapore. Association for Computational Linguistics

    doi:10.18653/v1/2023.emnlp-main.399 2023

  7. [7]

    Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.618 Translation artifacts in cross-lingual transfer learning . In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7674--7684, Online. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.618 2020

  8. [8]

    Laurie Burchell, Alexandra Birch, Nikolay Bogoychev, and Kenneth Heafield. 2023. https://doi.org/10.18653/v1/2023.acl-short.75 An open dataset and model for language identification . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 865--879, Toronto, Canada. Association for Computat...

    doi:10.18653/v1/2023.acl-short.75 2023

Show all 54 references
  1. [9]

    Tingfeng Cao, Chengyu Wang, Chuanqi Tan, Jun Huang, and Jinhui Zhu. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.33 Sharing, teaching and aligning: Knowledgeable transfer learning for cross-lingual machine reading comprehension . In Findings of the Association for Com...

    2023 doi

  2. [10]

    Yang Chen, Chao Jiang, Alan Ritter, and Wei Xu. 2023. https://doi.org/10.18653/v1/2023.findings-acl.357 Frustratingly easy label projection for cross-lingual transfer . In Findings of the Association for Computational Linguistics: ACL 2023, pages 5775--5796, Toronto, Canada. A...

    2023 doi

  3. [11]

    Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki

    Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. https://doi.org/10.1162/tacl_a_00317 T y D i QA : A benchmark for information-seeking question answering in typologically diverse languages . Transac...

    2020 doi

  4. [12]

    Simone Conia, Daniel Lee, Min Li, Umar Farooq Minhas, Saloni Potdar, and Yunyao Li. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.914 Towards cross-cultural machine translation with retrieval-augmented generation from multilingual knowledge graphs . In Proceedings of the 2...

    2024 doi

  5. [13]

    Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzm \'a n, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. https://doi.org/10.18653/v1/2020.acl-main.747 Unsupervised cross-lingual representation learning ...

    2020 doi

  6. [14]

    Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. https://doi.org/10.18653/v1/D18-1269 XNLI : Evaluating cross-lingual sentence representations . In Proceedings of the 2018 Conference on Empirical Methods ...

    2018 doi

  7. [15]

    Emilio Cueva, Adrian Lopez Monroy, Fernando S \'a nchez-Vega, and Thamar Solorio. 2024. https://doi.org/10.18653/v1/2024.naacl-long.460 Adaptive cross-lingual text classification through in-context one-shot demonstrations . In Proceedings of the 2024 Conference of the North Am...

    2024 doi

  8. [16]

    Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. https://openreview.net/forum?id=OUIFPHEgJU QL o RA : Efficient finetuning of quantized LLM s . In Thirty-seventh Conference on Neural Information Processing Systems

    2023

  9. [17]

    Benedikt Ebing and Goran Glava s . 2024. https://doi.org/10.18653/v1/2024.naacl-long.298 To translate or not to translate: A systematic investigation of translation-based cross-lingual transfer to low-resource languages . In Proceedings of the 2024 Conference of the North Amer...

    2024 doi

  10. [18]

    Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, and Jingjing Liu. 2021. https://doi.org/10.1609/aaai.v35i14.17512 Filter: An enhanced fusion method for cross-lingual language understanding . Proceedings of the AAAI Conference on Artificial Intelligence, 35(14):12776--12784

    2021 doi

  11. [19]

    Gemma Team , Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le La...

    2024 arXiv

  12. [20]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  13. [21]

    Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. https://proceedings.mlr.press/v97/houlsby19a.html Parameter-efficient transfer learning for NLP . In Proceedings of the 36th In...

    2019

  14. [22]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  15. [23]

    Sunkyoung Kim, Dayeon Ki, Yireun Kim, and Jinsik Lee. 2024. https://openreview.net/forum?id=CzcCUzJQER Cross-lingual QA : A key to unlocking in-context cross-lingual performance . In ICML 2024 Workshop on In-Context Learning

    2024

  16. [24]

    Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki M. Asano. 2024. https://openreview.net/forum?id=NjNfLdxr3A VeRA: V ector-based random matrix adaptation . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024

    2024

  17. [25]

    Anne Lauscher, Vinit Ravishankar, Ivan Vuli \'c , and Goran Glava s . 2020. https://doi.org/10.18653/v1/2020.emnlp-main.363 From zero to hero: O n the limitations of zero-shot language transfer with multilingual T ransformers . In Proceedings of the 2020 Conference on Empirica...

    2020 doi

  18. [26]

    McCarthy

    En-Shiun Annie Lee, Sarubi Thillainathan, Shravan Nayak, Surangika Ranathunga, David Ifeoluwa Adelani, Ruisi Su, and Arya D. McCarthy. 2022 a . https://doi.org/10.18653/v1/2022.findings-acl.6 Pre-trained multilingual sequence-to-sequence models: A hope for low-resource languag...

    2022 doi

  19. [27]

    Jaeseong Lee, Seung-won Hwang, and Taesup Kim. 2022 b . https://doi.org/10.18653/v1/2022.aacl-short.8 FAD - X : Fusing adapters for cross-lingual transfer to low-resource languages . In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Comput...

    2022 doi

  20. [28]

    o rg Tiedemann, Andr \' e F. T. Martins, and Hinrich Sch \

    Peiqin Lin, Shaoxiong Ji, J \" o rg Tiedemann, Andr \' e F. T. Martins, and Hinrich Sch \" u tze. 2024. https://doi.org/10.48550/arXiv.2401.13303 MaLA-500: M assive language adaptation of large language models . Preprint, arXiv:2401.13303

    2024 doi arXiv

  21. [29]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. https://proceedings.neurips.cc/paper_files/paper/2023/file/6dcf277ea32ce3288914faf369fe6de0-Paper-Conference.pdf Visual instruction tuning . In Advances in Neural Information Processing Systems, volume 36, pages 34...

    2023

  22. [30]

    Shih - Yang Liu, Chien - Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu - Chiang Frank Wang, Kwang - Ting Cheng, and Min - Hung Chen. 2024. https://openreview.net/forum?id=3d5CIRG1n2 Dora: Weight-decomposed low-rank adaptation . In Forty-first International Conference on Machine Lea...

    2024

  23. [31]

    NLLB Team , Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Pran...

    2022 arXiv

  24. [32]

    Jaehoon Oh, Jongwoo Ko, and Se-Young Yun. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.452 Synergy with translation artifacts for training and inference in multilingual tasks . In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, page...

    2022 doi

  25. [33]

    Jonas Pfeiffer, Sebastian Ruder, Ivan Vulic, and Edoardo M. Ponti. 2023. https://openreview.net/forum?id=z9EkXfvxta Modular deep learning . Transactions on Machine Learning Research, 2023

    2023

  26. [34]

    Jonas Pfeiffer, Ivan Vuli \'c , Iryna Gurevych, and Sebastian Ruder. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.617 MAD-X : A n A dapter- B ased F ramework for M ulti- T ask C ross- L ingual T ransfer . In Proceedings of the 2020 Conference on Empirical Methods in Natur...

    2020 doi

  27. [35]

    Ayu Purwarianti and Ida Ayu Putu Ari Crisdayanti. 2019. https://ieeexplore.ieee.org/document/8904199 Improving bi-lstm performance for indonesian sentiment analysis using paragraph vector . In 2019 International Conference of Advanced Informatics: Concepts, Theory and Applicat...

    2019

  28. [36]

    Tingyu Qu, Tinne Tuytelaars, and Marie-Francine Moens. 2025. https://www.ecva.net/papers/eccv_2024/papers_ECCV/papers/12192.pdf Introducing routing functions to vision-language parameter-efficient fine-tuning with low-rank bottlenecks . In Computer Vision -- ECCV 2024, pages 2...

    2025

  29. [37]

    Kiran Ramnath, Leda Sari, Mark Hasegawa-Johnson, and Chang Yoo. 2021. https://doi.org/10.18653/v1/2021.naacl-main.153 Worldly wise ( W o W ) - cross-lingual knowledge fusion for fact-based visual spoken-question answering . In Proceedings of the 2021 Conference of the North Am...

    2021 doi

  30. [38]

    Evgeniia Razumovskaia, Ivan Vuli\' c , and Anna Korhonen. 2024. https://doi.org/10.48550/arXiv.2403.01929 Analyzing and adapting large language models for few-shot multilingual NLU: are we there yet? Preprint, arXiv:2403.01929

    2024 doi arXiv

  31. [39]

    Sebastian Ruder, Ivan Vuli\' c , and Anders S gaard. 2019. https://doi.org/10.1613/jair.1.11640 A survey of cross-lingual word embedding models . J. Artif. Int. Res., 65(1):569–630

    2019 doi

  32. [40]

    Fabian David Schmidt, Philipp Borchert, Ivan Vuli \'c , and Goran Glava s . 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.394 Self-distillation for model stacking unlocks cross-lingual NLU in 200+ languages . In Findings of the Association for Computational Linguistics...

    2024 doi

  33. [41]

    Lei Shi, Rada Mihalcea, and Mingjun Tian. 2010. https://aclanthology.org/D10-1103/ Cross language text classification by model translation and semi-supervised learning . In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1057--1067...

    2010

  34. [42]

    Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. https://doi.org/10.18653/v1/2024.acl-long.309 Language-specific neurons: The key to multilingual capabilities in large language models . In Proceedings of the 62nd...

    2024 doi

  35. [43]

    Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur, and Tanmoy Chakraborty. 2023. https://doi.org/10.18653/v1/2023.acl-long.346 Multilingual LLM s are better cross-lingual in-context learners with alignment . In Proceedings of the 61st Annual Meeting of the Association for Comp...

    2023 doi

  36. [44]

    Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, and Jianfeng Gao. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.388 A da M ix: Mixture-of-adaptations for parameter-efficient model tuning . In Proceedings of the 2022 Confere...

    2022 doi

  37. [45]

    Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. https://doi.org/10.18653/v1/2024.acl-long.820 Do llamas work in E nglish? on the latent language of multilingual transformers . In Proceedings of the 62nd Annual Meeting of the Association for Computati...

    2024 doi

  38. [46]

    Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. https://doi.org/10.18653/v1/N18-1101 A broad-coverage challenge corpus for sentence understanding through inference . In Proceedings of the 2018 Conference of the North A merican Chapter of the Association for Computation...

    2018 doi

  39. [47]

    Genta Winata, Shijie Wu, Mayank Kulkarni, Thamar Solorio, and Daniel Preotiuc-Pietro. 2022. https://doi.org/10.18653/v1/2022.aacl-main.59 Cross-lingual few-shot learning on unseen languages . In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association f...

    2022 doi

  40. [48]

    Genta Indra Winata, Alham Fikri Aji, Samuel Cahyawijaya, Rahmad Mahendra, Fajri Koto, Ade Romadhony, Kemal Kurniawan, David Moeljadi, Radityo Eko Prasojo, Pascale Fung, Timothy Baldwin, Jey Han Lau, Rico Sennrich, and Sebastian Ruder. 2023. https://doi.org/10.18653/v1/2023.eac...

    2023 doi

  41. [49]

    Shijie Wu and Mark Dredze. 2020. https://doi.org/10.18653/v1/2020.repl4nlp-1.16 Are all languages created equal in multilingual BERT ? In Proceedings of the 5th Workshop on Representation Learning for NLP, pages 120--130, Online. Association for Computational Linguistics

    2020 doi

  42. [50]

    Shaoyang Xu, Junzhuo Li, and Deyi Xiong. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.226 Language representation projection: Can we transfer factual knowledge across languages in multilingual language models? In Proceedings of the 2023 Conference on Empirical Methods in ...

    2023 doi

  43. [51]

    Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. https://doi.org/10.18653/v1/2021.naacl-main.41 m T 5: A massively multilingual pre-trained text-to-text transformer . In Proceedings of the 2021 Conferenc...

    2021 doi

  44. [52]

    Huiyun Yang, Huadong Chen, Hao Zhou, and Lei Li. 2022. https://openreview.net/forum?id=OjPmfr9GkVv Enhancing cross-lingual transfer by manifold mixup . In International Conference on Learning Representations

    2022

  45. [53]

    Mengjie Zhao, Yi Zhu, Ehsan Shareghi, Ivan Vuli \'c , Roi Reichart, Anna Korhonen, and Hinrich Sch \"u tze. 2021. https://doi.org/10.18653/v1/2021.acl-long.447 A closer look at few-shot crosslingual transfer: The choice of shots matters . In Proceedings of the 59th Annual Meet...

    2021 doi

  46. [54]

    Yiran Zhao, Wenxuan Zhang, Huiming Wang, Kenji Kawaguchi, and Lidong Bing. 2024. https://arxiv.org/abs/2402.18913 AdaMergeX: C ross-lingual transfer with large language models via adaptive adapter merging . Preprint, arXiv:2402.18913

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.