Pith. sign in

REVIEW 4 major objections 5 minor 67 references

Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphs

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A 37K-parameter transducer rivals full fine-tuning on code tasks

desk verdict A genuinely new parameter-efficient adaptation idea, but the evaluation numbers look miscalibrated and the graph contribution is not actually demonstrated. read the letter →

arxiv 2412.13467 v1 pith:3FO5R3GA submitted 2024-12-18 cs.SE cs.AIcs.CL

Imam Nur Bani Yusuf , Lingxiao Jiang This is my paper · ORCID
classification cs.SEcs.AIcs.CL
keywords TransducerTuningCodePropertyGraphparameter-efficientfine-tuningfrozenbackbonemodelsummarizationassertgenerationtranslationneuralnetwork
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Transducer Tuning is a method for adapting a large code model to downstream software tasks while keeping the model itself frozen. The paper argues that a small trainable module, the Transducer, can inject structural and dependency information from Code Property Graphs into the model's input embeddings, so that a handful of new parameters carries the task adaptation. On code summarization, assert generation, and code translation, the authors report scores close to full fine-tuning while using only 30,700 to 37,100 trainable parameters, a reduction of up to 99 percent. They also report competitive results against LoRA, prompt tuning, and prefix tuning while using a fraction of those methods' trainable parameters. The intended payoff is that resource-constrained users can adapt ever-larger code models without the memory cost of updating the whole model.

What carries the argument

The load-bearing object is the Transducer, a two-part module inserted between the frozen backbone's embedding layer and its encoder and decoder. The Graph Vectorization Engine (GVE) turns each input's Code Property Graph into a single fixed-size vector: node labels are embedded, normalized, down-projected, propagated through a graph attention network, up-projected, and mean-pooled. The Attention-Based Fusion Layer (ABFL) then mixes that graph vector into the backbone's token embeddings by treating the graph vector as key and value, the code embedding as query, and applying a final projection back to the backbone's hidden dimension. Only these two components are updated during training, which is what keeps the parameter count near 30K while still allowing task-specific behavior.

What would settle it

Train the Attention-Based Fusion Layer alone with the same frozen backbone, same data, and same training budget, but replace the graph vector with a constant or random vector; if scores match the full Transducer on all three tasks, the CPG content is not carrying the reported gains.

Watch

Extended reading notes

Core claim

The central claim is that a frozen CodeT5+ model can be adapted to code summarization, assert statement generation, and code-to-code translation by training only the Transducer. The Transducer first builds a Code Property Graph, an amalgam of abstract syntax tree, control-flow graph, and program dependence graph, for each input method using static analysis. A Graph Vectorization Engine maps node labels to vectors, processes them with a graph attention network, and pools them into one graph feature vector; an Attention-Based Fusion Layer then uses that vector as keys and values while the model's own code embedding supplies the query, producing an enriched embedding that feeds the frozen encoder and decoder. The authors report that this reaches scores within about 0.1 to 2.9 points of full fine-tuning and other efficient methods on the 220M and 770M backbones, while using only 30.7K to 37.1K trainable parameters. They frame the result as evidence that injecting CPG structure and dependency information is a viable route to parameter-efficient model adaptation.

Load-bearing premise

The method's value rests on the premise that the graph-derived CPG information, not just the extra trainable attention layer, is what brings the frozen model up to task performance; if a graph-free fusion layer performs equally well, the approach reduces to a generic embedding adapter.

Editorial extensions

If this is right

  • Task adaptation reduces to training one small Transducer, so new downstream tasks can be added by training another Transducer on the same frozen backbone without changing or retraining the base model.
  • Memory for gradients and optimizer states shrinks to roughly the Transducer's size, which is what enables the reported 99 percent parameter reduction and makes the 770M model feasible on a 24GB GPU.
  • The method directly competes with LoRA, prefix tuning, and prompt tuning, using 1.5 to 80 percent of their trainable parameters while staying within roughly 0.1 to 2.9 points of their scores in the reported settings.
  • If the claims hold, parameter-efficient adaptation for code models no longer forces a stark choice between memory savings and accuracy: the reported gains over no fine-tuning, 2.13 to 10.21 points, are larger than the reported gaps to full fine-tuning, up to 2.9 points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own ablation shows a graph-free ABFL-only variant beating the full Transducer on CodeT5+ 770M for summarization and assert generation, so a fair reading is that some of the gain may come from the attention-based fusion adapter rather than from the CPG content itself.
  • A natural next test would replace CPG-derived vectors with random or sequence-only features inside the same Transducer architecture; equal performance would suggest the graph content is not the active ingredient.
  • Because the Transducer only requires a graph with labeled nodes, the recipe likely transfers to non-code graphs such as knowledge graphs or social networks, a generalization the paper itself suggests.
  • The reported 770M results show larger gaps to full fine-tuning than the 220M results, so the competitiveness claim may weaken as the backbone grows; whether the trade-off remains acceptable at billion-parameter scale is left open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Transducer Tuning, a parameter-efficient adaptation method for code LLMs. A Transducer module consisting of a Graph Vectorization Engine (GVE) and an Attention-Based Fusion Layer (ABFL) extracts Code Property Graphs (CPGs) from input code, converts them into graph feature vectors, and fuses them with the frozen backbone model's input embeddings. The method is evaluated on code summarization, assert generation, and code translation with CodeT5+ 220M and 770M, comparing against full fine-tuning, LoRA, prefix-tuning, prompt-tuning, and a linear adapter. The central claims are that Transducer Tuning is competitive with full fine-tuning while reducing trainable parameters by up to 99% (using 30.7K-37.1K parameters), and that the injection of CPG structural/dependency information is what enables this efficient adaptation.

Significance. If the central claims were supported, the work would provide a practically useful parameter-efficient adaptation method for code tasks, with the unusual property of injecting explicit graph structure into a frozen LLM. The paper has clear strengths: it releases source code and preprocessed datasets, the internal comparison is run consistently across methods, the reported parameter counts are credible, and the ablation compares the proposed method against its own graph-free variant. However, the headline results are presented in a metric regime that is not interpretable as reported, the test sets are filtered to nonstandard subsets, and the ablation undercuts the paper's core attribution of the gains to CPG information. These issues are load-bearing for the abstract's main claims, so the manuscript cannot be accepted in its current form.

major comments (4)
  1. [Section 5, Table 1; Section 4.3] The absolute scores in Table 1 are far outside the published range for these benchmarks. CodeT5+ 220M with no fine-tuning is reported at 95.49 smoothed BLEU on code summarization and 94.47 CodeBLEU on code translation, and almost all tuning methods sit between 96 and 99. For CodeXGLUE/CodeSearchNet Java summarization, CodeT5-class models typically score around 19-20 BLEU, so these numbers suggest either a miscalibrated evaluation script, a different metric than the standard CodeXGLUE BLEU/CodeBLEU, or a trivially easy filtered test set. The 'No Fine-tuning' baseline is also not described operationally in Section 4.3, and a near-ceiling score from the pre-trained state alone is unexplained. The central comparison cannot be assessed until the authors run the official CodeXGLUE evaluation scripts on the original test sets and report those results alongside any filtered-subset numbers.
  2. [Section 4.1 vs Appendix A, Tables A2-A3] The test-set sizes reported in Section 4.1 are inconsistent with Appendix A: the text says the assert-generation and code-translation test sets have 3.3K and 370 instances, respectively, while Tables A2 and A3 report 6262 and 896. If Appendix A reports pre-filter sizes and Section 4.1 post-filter sizes, this must be stated explicitly. More importantly, all headline results are obtained on filtered subsets that retain only 41%-53% of the original test instances for code-to-code tasks. The comparison to full fine-tuning is therefore on a nonstandard, potentially easier subset, and the paper should report performance on the original unfiltered test splits as well.
  3. [Section 6.1, Table 3] The ablation study directly contradicts the claim that CPG graph information is the source of the gains. For CodeT5+ 770M, the graph-free ABFL-only variant outperforms the full GVE+ABFL Transducer on code summarization (98.64 vs 98.11) and on assert generation (83.16 vs 81.16), with the ABFL-only assert-generation score exactly matching full fine-tuning. This is not a 'minimal performance difference' on assert generation; it is a 2.0-point gap in favor of the variant without any graph features. The paper's framing that CPG structural and dependency information 'enhances model performance' is not supported by these results, and the authors need either a substantially stronger analysis of when graph information helps or a revised framing that does not attribute the method's performance to CPGs.
  4. [Section 5, Takeaway 2; Section 6.4] The paper's argument that relative differences matter more than absolute scores does not address the saturating-metric problem. In a range where no-fine-tuning baselines already score 87-95, a 1-3 point gap between methods is not a meaningful demonstration of competitiveness. The threats-to-validity discussion cannot rescue the headline comparison; the evaluation needs to be redone on a metric with headroom, or the conclusions must be substantially scaled back.
minor comments (5)
  1. [Section 4.1, Appendix A] Please state explicitly whether the dataset statistics in Appendix A are pre-filter or post-filter; the current text creates an apparent contradiction between the test sizes in Section 4.1 and Tables A2 and A3.
  2. [Table 3 caption] The caption says the table compares 'GVE + ABFL' with 'three other variants: GVE-only, ABFL-only, and Linear,' but no Linear row appears in the table; the caption should list only the variants actually reported.
  3. [Section 3.1.2, Equation (2)] Please define the shapes of Q, K, and V and clarify that the attention is computed between the sequence of token embeddings and a single graph feature vector, since the current notation suggests Q, K, and V all have the same dimensions.
  4. [Appendix C.3.3] There is a typo: 'an there is an option' should read 'and there is an option.'
  5. [References] The reference entry 'et al., S.L. (2021). CodeXGLUE' is malformed and should be replaced with the full author list or a standard abbreviated citation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the empirical claims rest on held-out evaluations, externally sourced graph embeddings, and direct parameter counts rather than on self-citation or definitional equivalence.

full rationale

The paper's derivation chain is empirical and self-contained. The Transducer is trained only on designated training splits, evaluated on held-out test splits, and its hyperparameters are selected using a validation split (Section 4.3, Appendix B). The central performance claim compares test-set metrics of Transducer Tuning against baselines run under the same evaluation protocol, so the comparison is not a fitted-parameter-renamed-as-prediction. The graph feature vectors come from an external embedding model (mxbai-embed-large-v1) via the GVE, and the backbone CodeT5+ models are frozen; nothing in the architecture defines a target metric in terms of its own output. Parameter counts (Table 2) are direct architectural counts, and the reported reductions (up to 99%, and 1.5%-80% of other methods' parameters) are arithmetic consequences of those counts. The authors' self-citations (Yusuf et al. 2022, 2023) appear only as examples of code-related tasks or as supporting references for standard BLEU/CodeBLEU metrics; they are not load-bearing for the central claim. The ablation study (Table 3) reveals that graph information helps in only 4 of 6 settings, which weakens the interpretive claim that CPG structure drives the gains, but that is a correctness or internal-validity concern, not circularity. No quoted passage in the paper exhibits a definitional reduction (Equation X = Equation Y by construction) or a fitted input renamed as a prediction. Hence the paper does not display circular reasoning.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The central claim relies on the assumption that CPG structure improves adaptation, which the paper's ablation only partially confirms, plus several component-level domain assumptions. No free parameters are fitted to the test set; the listed hand-set dimensions and the 50-node truncation cap shape the parameter count and performance.

free parameters (3)
  • Down-projection dimension d_down = 8
    Hand-chosen, inspired by LoRA rank from prior work (Section 4.2); directly determines the trainable parameter count of the Graph Processing Layer.
  • Attention fusion dimension d_abf = 8
    Hand-chosen (Section 4.2); determines the size of Q/K/V projections in ABFL and contributes to the parameter count.
  • Maximum CPG nodes per method = 50
    Dataset statistics in Appendix A show max node count is 50 for all splits, implying CPGs are truncated; this cap limits the structural information available and is not reported as tuned.
assumptions (5)
  • domain assumption Code Property Graphs capture syntactic structure, control flow, and data dependencies that are useful for code understanding.
    Invoked in Sections 2 and 6.3 to justify using CPGs; the paper's own ablation (Table 3) only partially supports this for the two larger-model tasks.
  • domain assumption Joern produces faithful CPGs for the Java and C# methods in the datasets.
    Used to extract graphs for all inputs (Section 4.1); no verification of extraction failures or partial graphs is reported.
  • domain assumption The generic text embedding model mxbai-embed-large-v1 provides node-label vectors of sufficient quality for graph vectorization.
    Chosen for MTEB performance and small size (Section 4.1); no code-specific evaluation of these embeddings is given.
  • domain assumption A single mean-pooled graph vector G plus attention over token embeddings is sufficient to inject structural information into the backbone.
    Core architectural premise of ABFL (Section 3.1.2); simplification not validated against richer fusion mechanisms.
  • standard math Scaled dot-product attention (Vaswani et al., 2017) and GNN message passing are well-defined operations that can be optimized by gradient descent.
    Standard tools used in the architecture (Equations 1 and 2).
invented entities (1)
  • Transducer module (GVE + ABFL)
    purpose: A trainable input-side component that enriches frozen backbone embeddings with CPG-derived graph features for task adaptation.
    The module's effectiveness is evaluated only on this paper's three tasks and two CodeT5+ backbones; no independent external validation or downstream adoption is cited. The released code could enable external testing, but no such evidence is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphs." pith.science (2026). https://pith.science/paper/3FO5R3GA

@misc{pith2026241213467,
  author       = {Pith},
  title        = {Pith review of: Transducer Tuning: Efficient Model Adaptation for Software Tasks Using Code Property Graphs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3FO5R3GA}},
  note         = {Machine review of arXiv:2412.13467}
}
read the original abstract

Large language models have demonstrated promising performance across various software engineering tasks. While fine-tuning is a common practice to adapt these models for downstream tasks, it becomes challenging in resource-constrained environments due to increased memory requirements from growing trainable parameters in increasingly large language models. We introduce \approach, a technique to adapt large models for downstream code tasks using Code Property Graphs (CPGs). Our approach introduces a modular component called \transducer that enriches code embeddings with structural and dependency information from CPGs. The Transducer comprises two key components: Graph Vectorization Engine (GVE) and Attention-Based Fusion Layer (ABFL). GVE extracts CPGs from input source code and transforms them into graph feature vectors. ABFL then fuses those graphs feature vectors with initial code embeddings from a large language model. By optimizing these transducers for different downstream tasks, our approach enhances the models without the need to fine-tune them for specific tasks. We have evaluated \approach on three downstream tasks: code summarization, assert generation, and code translation. Our results demonstrate competitive performance compared to full parameter fine-tuning while reducing up to 99\% trainable parameters to save memory. \approach also remains competitive against other fine-tuning approaches (e.g., LoRA, Prompt-Tuning, Prefix-Tuning) while using only 1.5\%-80\% of their trainable parameters. Our findings show that integrating structural and dependency information through Transducer Tuning enables more efficient model adaptation, making it easier for users to adapt large models in resource-constrained settings.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 59 canonical work pages

  1. [1]

    Ahmad, W.U., Chakraborty, S., Ray, B., Chang, K. (2021). Unified pre-training for program understanding and generation. NAACL-HLT (pp. 2655–2668). ACL

    work page 2021

  2. [2]

    Ahmed, T., Pai, K.S., Devanbu, P., Barr, E. (2024). Automatic semantic augmentation of language model prompts (for code summarization). ICSE. ACM

    work page 2024

  3. [3]

    Allamanis, M., Brockschmidt, M., Khademi, M. (2018). Learning to represent programs with graphs. ICLR. OpenReview.net

    work page 2018

  4. [4]

    Ba, L.J., Kiros, J.R., Hinton, G.E. (2016). Layer normalization. ArXiv e-prints , arXiv–1607, 15

    work page 2016

  5. [5]

    Bapna, A., & Firat, O. (2019). Simple, scalable adaptation for neural machine translation. EMNLP/IJCNLP (1) (pp. 1538–1548). ACL

    work page 2019

  6. [6]

    Brody, S., Alon, U., Yahav, E. (2022). How attentive are graph attention networks? ICLR. OpenReview.net

    work page 2022

  7. [7]

    Chakraborty, S., Krishna, R., Ding, Y., Ray, B. (2022). Deep learning based vulnerability detection: Are we there yet? IEEE TSE , 48 (9), 3280–3296,

    work page 2022

  8. [8]

    Devlin, J., Chang, M., Lee, K., Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. NAACL-HLT (1) (pp. 4171–4186). ACL

    work page 2019

Show all 67 references
  1. [9]

    Dey, S., Vinayakarao, V., Gupta, M., Dechu, S. (2022). Evaluating commit message generation: To BLEU or not to bleu? ICSE (NIER) (pp. 31–35). IEEE/ACM. et al., S.L. (2021). CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. NeurIPS datasets ...

    2022

  2. [10]

    Feng, Z., Guo, D., Tang, D., Duan, N., Feng, X., Gong, M., . . . Zhou, M. (2020). Codebert: A pre-trained model for programming and natural languages.EMNLP (findings) (Vol. EMNLP 2020, pp. 1536–1547). ACL

    2020

  3. [11]

    Fu, M., Tantithamthavorn, C., Le, T., Nguyen, V., Phung, D.Q. (2022). Vulrepair: a t5-based automated software vulnerability repair. ESEC/SIGSOFT FSE (pp. 935–947). ACM

    2022

  4. [12]

    Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Liu, S., . . . Zhou, M. (2021). Graphcodebert: Pre-training code representations with data flow. ICLR. OpenReview.net

    2021

  5. [13]

    Han, J., Huang, C., Sun, S., Liu, Z., Liu, J. (2023). bjxnet: an improved bug local- ization model based on code property graph and attention mechanism. Autom. Softw. Eng. , 30 (1), 12,

    2023

  6. [14]

    Han, Z., Gao, C., Liu, J., Zhang, J., Zhang, S.Q. (2024). Parameter-efficient fine-tuning for large models: A comprehensive survey. CoRR, abs/2403.14608 , ,

    2024 arXiv

  7. [15]

    Gelly, S

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., . . . Gelly, S. (2019). Parameter-efficient transfer learning for NLP. ICML (Vol. 97, pp. 2790–2799). PMLR

    2019

  8. [16]

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., . . . Chen, W. (2022). Lora: Low-rank adaptation of large language models. ICLR. OpenReview.net. 16

    2022

  9. [17]

    Hu, X., Li, G., Xia, X., Lo, D., Jin, Z. (2020). Deep code comment generation with hybrid lexical and syntactical information. Empir. Softw. Eng. , 25 (3), 2179–2217,

    2020

  10. [18]

    Husain, H., Wu, H., Gazit, T., Allamanis, M., Brockschmidt, M. (2019). CodeSearch- Net challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436 , ,

    2019 arXiv

  11. [19]

    Hyeon-Woo, N., Ye-Bin, M., Oh, T. (2022). Fedpara: Low-rank hadamard product for communication-efficient federated learning. ICLR. OpenReview.net

    2022

  12. [20]

    Jiang, N., Lutellier, T., Tan, L. (2021). CURE: code-aware neural machine translation for automatic program repair. ICSE (pp. 1161–1173). IEEE

    2021

  13. [21]

    Jin, M., Shahriar, S., Tufano, M., Shi, X., Lu, S., Sundaresan, N., Svyatkovskiy, A. (2023). Inferfix: End-to-end program repair with llms. ESEC/SIGSOFT FSE (pp. 1646–1656). ACM

    2023

  14. [22]

    Zhang, M

    Ju, W., Fang, Z., Gu, Y., Liu, Z., Long, Q., Qiao, Z., . . . Zhang, M. (2024). A comprehensive survey on deep graph representation learning. Neural Networks , ,

    2024

  15. [23]

    Kopiczko, D.J., Blankevoort, T., Asano, Y.M. (2024). Vera: Vector-based random matrix adaptation. ICLR. OpenReview.net

    2024

  16. [24]

    Lester, B., Al-Rfou, R., Constant, N. (2021). The power of scale for parameter-efficient prompt tuning. EMNLP (1) (pp. 3045–3059). ACL

    2021

  17. [25]

    Li, X.L., & Liang, P. (2021). Prefix-tuning: Optimizing continuous prompts for generation. ACL/IJCNLP (1) (pp. 4582–4597). ACL

    2021

  18. [26]

    Lin, C., & Och, F.J. (2004). ORANGE: a method for evaluating automatic evaluation metrics for machine translation. COLING

    2004

  19. [27]

    Liu, H., Tam, D., Muqeeth, M., Mohta, J., Huang, T., Bansal, M., Raffel, C. (2022). Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. NeurIPS

    2022

  20. [28]

    Liu, J., Sha, C., Peng, X. (2023). An empirical study of parameter-efficient fine-tuning methods for pre-trained code models. ASE (pp. 397–408). IEEE

    2023

  21. [29]

    Liu, J., Zeng, J., Wang, X., Liang, Z. (2023). Learning graph-based code repre- sentations for source-level functional similarity detection. ICSE (pp. 345–357). IEEE. 17

    2023

  22. [30]

    Liu, R., Wang, Y., Xu, H., Liu, B., Sun, J., Guo, Z., Ma, W. (2024). Source code vulnerability detection: Combining code language models and code property graphs. CoRR, abs/2404.14719 , ,

    2024 arXiv

  23. [31]

    Liu, X., Ji, K., Fu, Y., Tam, W., Du, Z., Yang, Z., Tang, J. (2022). P-tuning: Prompt tuning can be comparable to fine-tuning across scales and tasks. ACL (2) (pp. 61–68)

    2022

  24. [32]

    Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. ICLR (poster). OpenReview.net

    2019

  25. [33]

    Jiang, D

    Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., . . . Jiang, D. (2024). Wiz- ardcoder: Empowering code large language models with evol-instruct. ICLR. OpenReview.net

    2024

  26. [34]

    Oliveto, R., Bavota, G. (2021). Studying the usage of text-to-text transfer transformer to support code-related tasks. ICSE (pp. 336–347). IEEE

    2021

  27. [35]

    Mi, Q., Zhan, Y., Weng, H., Bao, Q., Cui, L., Ma, W. (2023). A graph-based code representation method to improve code readability classification. Empir. Softw. Eng., 28 (4), 87,

    2023

  28. [36]

    Longpre, S

    Muennighoff, N., Liu, Q., Zebaze, A.R., Zheng, Q., Hui, B., Zhuo, T.Y., . . . Longpre, S. (2024). Octopack: Instruction tuning code large language models. ICLR. OpenReview.net

    2024

  29. [37]

    Muennighoff, N., Tazi, N., Magne, L., Reimers, N. (2023). MTEB: massive text embedding benchmark. EACL (pp. 2006–2029). ACL

    2023

  30. [38]

    Niu, C., Li, C., Ng, V., Chen, D., Ge, J., Luo, B. (2023). An empirical comparison of pre-trained models of source code. ICSE (pp. 2136–2148). IEEE

    2023

  31. [39]

    Papineni, K., Roukos, S., Ward, T., Zhu, W. (2002). Bleu: a method for automatic evaluation of machine translation. ACL (pp. 311–318). ACL

    2002

  32. [40]

    Pfeiffer, J., Kamath, A., R¨ uckl´ e, A., Cho, K., Gurevych, I. (2021). Adapterfusion: Non-destructive task composition for transfer learning. EACL (pp. 487–503). ACL

    2021

  33. [41]

    Gurevych, I

    Pfeiffer, J., R¨ uckl´ e, A., Poth, C., Kamath, A., Vulic, I., Ruder, S., . . . Gurevych, I. (2020). Adapterhub: A framework for adapting transformers. EMNLP (demos) (pp. 46–54). ACL. 18

    2020

  34. [42]

    Pfeiffer, J., Vulic, I., Gurevych, I., Ruder, S. (2020). MAD-X: an adapter-based framework for multi-task cross-lingual transfer. EMNLP (1) (pp. 7654–7673). ACL

    2020

  35. [43]

    Ponti, E.M., Sordoni, A., Reddy, S. (2022). Combining modular skills in multitask learning. CoRR, abs/2202.13914 , ,

    2022 arXiv

  36. [44]

    Ren, S., Guo, D., Lu, S., Zhou, L., Liu, S., Tang, D., . . . Ma, S. (2020). Codebleu: a method for automatic evaluation of code synthesis. CoRR, abs/2009.10297 , ,

    2020 arXiv

  37. [45]

    Rumelhart, D.E., Hinton, G.E., Williams, R.J. (1986). Learning representations by back-propagating errors. nature, 323 (6088), 533–536,

    1986

  38. [46]

    Shi, E., Wang, Y., Du, L., Chen, J., Han, S., Zhang, H., . . . Sun, H. (2022). On the evaluation of neural code summarization. ICSE (pp. 1597–1608). ACM

    2022

  39. [47]

    Shi, E., Wang, Y., Zhang, H., Du, L., Han, S., Zhang, D., Sun, H. (2023). Towards effi- cient fine-tuning of pre-trained code models: An experimental study and beyond. ISSTA (pp. 39–51). ACM

    2023

  40. [48]

    Tian, H., Liu, K., Kabor´ e, A.K., Koyuncu, A., Li, L., Klein, J., Bissyand´ e, T.F. (2020). Evaluating representation learning of code changes for predicting patch correctness in program repair. ASE (pp. 981–992). IEEE

    2020

  41. [49]

    Polosukhin, I. (2017). Attention is all you need. NIPS (pp. 5998–6008)

    2017

  42. [50]

    Wan, Y., He, Y., Bi, Z., Zhang, J., Zhang, H., Sui, Y., . . . Yu, P.S. (2024). Deep learn- ing for code intelligence: Survey, benchmark and toolkit.CoRR, abs/2401.00288 , ,

    2024 arXiv

  43. [51]

    Wang, Y., Le, H., Gotmare, A., Bui, N.D.Q., Li, J., Hoi, S.C.H. (2023). Codet5+: Open code large language models for code understanding and generation. EMNLP (pp. 1069–1088). ACL

    2023

  44. [52]

    Watson, C., Tufano, M., Moran, K., Bavota, G., Poshyvanyk, D. (2020). On learning meaningful assert statements for unit test cases. ICSE (pp. 1398–1409). ACM

    2020

  45. [53]

    Wei, B., Li, G., Xia, X., Fu, Z., Jin, Z. (2019). Code generation as a dual task of code summarization. NeurIPS (pp. 6559–6569)

    2019

  46. [54]

    Weyssow, M., Zhou, X., Kim, K., Lo, D., Sahraoui, H.A. (2023). Exploring parameter- efficient fine-tuning techniques for code generation with large language models. 19 CoRR, abs/2308.10462 , ,

    2023 arXiv

  47. [55]

    Wu, L., Cui, P., Pei, J., Zhao, L., Guo, X. (2023). Graph neural networks: Foundation, frontiers and applications. KDD (pp. 5831–5832). ACM

    2023

  48. [56]

    Xia, C.S., Wei, Y., Zhang, L. (2023). Automated program repair in the era of large pre-trained language models. ICSE (pp. 1482–1494). IEEE

    2023

  49. [57]

    Xu, J., Sun, X., Zhang, Z., Zhao, G., Lin, J. (2019). Understanding and improving layer normalization. NeurIPS (pp. 4383–4393)

    2019

  50. [58]

    Yamaguchi, F., Golde, N., Arp, D., Rieck, K. (2014). Modeling and discovering vulnerabilities with code property graphs. IEEE symposium on security and privacy (pp. 590–604). IEEE Computer Society

    2014

  51. [59]

    Yeh, S., Hsieh, Y., Gao, Z., Yang, B.B.W., Oh, G., Gong, Y. (2024). Navigating text- to-image customization: From lycoris fine-tuning to model evaluation. ICLR. OpenReview.net

    2024

  52. [60]

    Yusuf, I.N.B., Jamal, D.B.A., Jiang, L. (2023). Automating arduino programming: From hardware setups to sample source code generation. MSR (pp. 453–464). IEEE

    2023

  53. [61]

    Yusuf, I.N.B., Jiang, L., Lo, D. (2022). Accurate generation of trigger-action programs with domain-adapted sequence-to-sequence learning. ICPC (pp. 99–110). ACM

    2022

  54. [62]

    Zan, D., Chen, B., Zhang, F., Lu, D., Wu, B., Guan, B., . . . Lou, J. (2023). Large language models meet nl2code: A survey. ACL (1) (pp. 7443–7464). ACL

    2023

  55. [63]

    Zhang, B., & Sennrich, R. (2019). Root mean square layer normalization. NeurIPS (pp. 12360–12371)

    2019

  56. [64]

    Deursen, A. (2023). Using large-scale heterogeneous graph representation learn- ing for code review recommendations at microsoft. ICSE-SEIP (pp. 162–172). IEEE

    2023

  57. [65]

    Zhang, K., Wang, W., Zhang, H., Li, G., Jin, Z. (2022). Learning to represent programs with heterogeneous graphs. ICPC (pp. 378–389). ACM

    2022

  58. [66]

    Zheng, T., Zhang, G., Shen, T., Liu, X., Lin, B.Y., Fu, J., . . . Yue, X. (2024). Open- codeinterpreter: Integrating code generation with execution and refinement. ACL (findings) (pp. 12834–12859). ACL

    2024

  59. [67]

    Zheng, Z., Ning, K., Wang, Y., Zhang, J., Zheng, D., Ye, M., Chen, J. (2023). A survey of large language models for code: Evolution, benchmarking, and future trends. CoRR, abs/2311.10372 , , 20 Declarations Author Contribution: The two authors discussed the original idea and r...

    2023 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.