Pith. sign in

REVIEW 3 major objections 5 minor 89 references

Natural Language-Programming Language Software Traceability Link Recovery Needs More than Textual Similarity

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that requirements-to-code traceability link recovery needs auxiliary strategies beyond textual similarity, with multi-strategy HGT and Gemini 2.5 Pro reporting average F1 gains of 3.68% and 8.84% over the previous best…

desk verdict Potentially useful benchmark and Difference Ratio metric, but the central F1 claims are untrustworthy as written because user-feedback links sampled from ground truth are never excluded from evaluation. read the letter →

arxiv 2509.05585 v1 pith:NMURC5AA submitted 2025-09-06 cs.SE cs.AI

Zhiyuan Zou , Bangchao Wang , Peng Liang , Tingting Bi , Huan Jin This is my paper · ORCID
classification cs.SEcs.AI
keywords softwaretraceabilitylinkrecoverytextsimilarityHeterogeneousGraphTransformerlargelanguagemodelrequirements-to-codeDifferenceRatioauxiliarystrategies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that textual similarity alone cannot reliably recover traceability links between natural-language requirements and code. Using a systematic review of 99 open-source projects and a new diagnostic metric, the Difference Ratio, it argues that NL-PL artifact pairs have far less lexical separation between true and false links than NL-NL or PL-PL pairs. To close that gap, it integrates three auxiliary strategies as heterogeneous-graph edge types in an HGT model and as additional prompt information in Gemini 2.5 Pro. On twelve requirements-to-code projects, the integrated models report average F1 gains of 3.68% and 8.84% over the previous best supervised baseline. A sympathetic reader would take the paper's contribution to be an empirical case that strategy integration, not better similarity alone, is what moves NL-PL traceability.

What carries the argument

The Difference Ratio, $(P_{\text{true}} - P_{\text{false}})/P_{\text{false}}$, compares average co-occurrence word ratios of true versus false link pairs; a low value diagnoses a dataset where lexical similarity cannot separate links. The recovery machinery then encodes auxiliary strategies as graph structure: code artifacts become nodes connected by import, extend, and method-call edges; requirement-code pairs get user-feedback edges from a random 10% of ground-truth links; and fine-grained semantic edges mark pairs where all seven decomposed code components (class name, method name, comments, parameters, return types) rank in the top 20% by VSM similarity to the requirement. HGT's type-aware attention learns the contribution of each edge type, while Gemini-All translates the same three strategies into sentences placed in the prompt's Additional_Information block.

What would settle it

Inspect the released replication package and compare the user-feedback link list used to build HGT-All edges and Gemini-All prompts against the labels used for the reported Precision, Recall, and F1; any overlap means the central performance claim is not supported. A cleaner check is to rerun the twelve-project evaluation with a strict split where feedback links are sampled only from the training set and never from the test set, and see whether the 3.68% and 8.84% average gains survive.

Watch

Extended reading notes

Core claim

The central claim is that NL-PL traceability link recovery is limited by a semantic gap that word-overlap measures cannot bridge, and that injecting domain-specific auxiliary strategies into powerful base models is an effective remedy. Concretely, the paper claims that HGT-All, which adds code-dependency, user-feedback, and fine-grained similarity edges to a Heterogeneous Graph Transformer, and Gemini-All, which feeds the same signals as text into a prompt-based LLM, both outperform their no-strategy counterparts and the previous best method HGNNLink. The paper reports average F1-score improvements of 3.68% for HGT-All and 8.84% for Gemini-All over HGNNLink across twelve open-source Java projects, with statistical significance at the 0.05 level.

Load-bearing premise

The load-bearing premise is that the 10% of ground-truth links randomly sampled as user feedback (Section 4.3.1, Eq. (17), and prompt Additional_Information 2) are not also used as evaluation labels; the manuscript never describes a train/test split or a hold-out of feedback links, so if those labels are visible to the models the reported gains are inflated by leakage.

Editorial extensions

If this is right

  • For NL-PL datasets, text-similarity-only methods should be treated as weak baselines; the Difference Ratio can serve as a cheap pre-screening test for whether auxiliary strategies are needed.
  • In supervised settings, HGT-All significantly outperforms GA-XWCoDe and HGNNLink at the 0.05 level across the twelve evaluated projects.
  • In unsupervised settings, Gemini-All significantly outperforms TAROT, with larger gains on small projects, while HGT-All wins on the largest projects.
  • New auxiliary strategies can be added downstream: HGT-All accommodates them as new edge types, and Gemini-All as new sentences in Additional_Information.
  • The paper's own implications note that HGT-All requires labeled training data and is sensitive to data scale, whereas Gemini-All is costly and sensitive to noisy strategy input, so the two cover complementary deployment scenarios.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the user-feedback edges are sampled from ground-truth links without a held-out test set, a portion of the evaluation labels may reach HGT-All and Gemini-All through feedback edges or prompt text; checking the released package for overlap between feedback links and scored labels would settle whether the reported 3.68% and 8.84% gains are real.
  • The Difference Ratio could be used by other researchers as a dataset-suitability prior: projects with low or negative ratio are candidates for multi-strategy integration, and one could test that hypothesis by predicting per-project F1 gains from ratio alone.
  • The edge-type recipe should transfer to other languages: replace the Java dependency extraction with Python or C++ dependency analysis and keep the HGT/Gemini prompts unchanged, giving a testable cross-language extension.
  • A practical hybrid not explored in the paper would run HGT-All on large repositories, run Gemini-All on small ones, and have Gemini verify HGT's borderline candidates, combining the two models' complementary strengths.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper argues that textual similarity alone is insufficient for natural language-to-programming language (NL-PL) traceability link recovery (TLR), and that auxiliary strategies are needed. To support this, the authors report a systematic literature review of 50 studies and 99 projects, propose a new metric called the Difference Ratio, and present a case study on requirements-to-code TLR. They integrate three auxiliary strategies (code dependency, user feedback, and fine-grained semantic similarity) into two models: a supervised Heterogeneous Graph Transformer (HGT-All) and a prompt-based Gemini 2.5 Pro model (Gemini-All). The abstract claims average F1-score improvements of 3.68% for HGT-All and 8.84% for Gemini-All over the state-of-the-art HGNNLink across twelve open-source projects. The paper also reports statistical significance tests, ablations, and includes a replication package.

Significance. If the empirical results were valid, the paper would make a useful contribution: it assembles a large project corpus, proposes a lightweight diagnostic metric, and demonstrates a principled way to inject domain knowledge into both graph-based and LLM-based TLR models. The replication package and the systematic organization of prior strategies are valuable resources for the community. However, the central quantitative claims are currently undermined by a likely label-leakage problem in the user-feedback strategy and by internal inconsistencies in the reported tables and observations. The paper's direction is plausible, but the evidence as presented does not support the headline numbers.

major comments (3)
  1. [Section 4.3.1 (Eq. 17) and Section 4.3.2 (Additional_Information 2)] The evaluation protocol for the user-feedback strategy is not sound as described. Eq. (17) states that HGT-All randomly samples 10% of the links from the ground-truth traceability set and adds each sampled positive link as a feedback edge. For Gemini-All, Additional_Information 2 tells the model 'User feedback indicates label is 0 or 1'. The paper never specifies a train/test split, a held-out set, or a statement that feedback links are excluded from evaluation. If any evaluated positive pair is among the sampled 10%, HGT-All receives a direct edge between exactly that pair and Gemini-All receives the answer in its prompt. Because only positive links are sampled, the leak is one-directional and would inflate recall and F1. This directly affects the abstract's claimed 3.68% and 8.84% improvements over HGNNLink. The authors must either show that feedback links were held out or re-run the experiments with an explicit split that prevents the model from seeing test labels.
  2. [Tables 7 and 9] The headline results cannot be reconstructed from the reported tables because of inconsistencies. HGT-All on Albergate is reported as Precision 0.4444, Recall 0.8000, F1 0.5714 in Table 7, but Table 9 gives Precision 0.4189, Recall 0.6200, F1 0.4914 for the same configuration and project. The Table 9 Albergate values are identical to the Table 7 HGT-Fine-grained row, suggesting a copying error. Additionally, the Pig and Seam2 rows in Table 9 contain the same numerical values for all methods (GA-XWCoDe, HGNNLink, HGT-All, Gemini-All), which is implausible for two different projects. These discrepancies must be resolved before the average F1 improvements in the abstract can be trusted.
  3. [Section 3.3.3, Observation 4] Observation 4 reverses the paper's own evidence. The RQ2 results in Table 3 show that NL-PL artifact pairs such as Requirements-Code have a low Difference Ratio (+93.86%) and low average F1 (0.3824), while NL-NL and PL-PL pairs such as Issue-Commit (+465.19%, F1 0.5602) and Test-Code (+885.19%, F1 0.5602) have higher ratios and better F1. Table 5 further shows that heterogeneous artifact pairs use enhancement strategies more often (67.08%) than homogeneous pairs (54.81%). Observation 4 states the opposite: that text similarity methods perform poorly for NL-NL/PL-PL tasks and that strategies are less necessary for NL-PL tasks. This contradiction also undermines the motivation in Section 4.2.1, which invokes Observation 4 to justify the case study. The observation should be rewritten to match the data.
minor comments (5)
  1. [Section 1] The paragraph beginning 'Second, due to the modular nature of PL artifacts' is duplicated verbatim.
  2. [Section 3.2.2 and Table 3] The metric is inconsistently named: the text and Table 3 use both 'Difference Ratio' and 'Different Ratio'. Please use one term throughout.
  3. [Section 4.3] The sentence 'HGT-All includes three components... (explained in Section??)' contains an unresolved cross-reference placeholder.
  4. [Section 4.3.1, Eqs. (28)-(31)] The final classifier is described inconsistently: Eq. (30) stops at f4, the text introduces f5, and Eq. (31) computes an inner product of r_i and c_j. Please clarify the exact architecture used in the experiments.
  5. [Section 3.3.3] The text contains several typos in the strategy examples, including 'examp;e', 'folows', 'isEqalTo', and 'TurePositives' in Eq. (1).

Circularity Check

2 steps flagged · score 6.0 of 10

User-feedback strategy leaks ground-truth labels into both models with no stated held-out split, so part of the reported F1 gains is a by-construction label echo.

  1. fitted input called prediction [Section 4.3.1, Eq. (17) (user feedback edges)]
    "To simulate real-world development scenarios where users provide partial feedback on trace links, we randomly sample 10% of the links from the ground truth traceability set. For each sampled positive link (r_i,c_j), we mark it as user feedback and add a feedback edge: e^{feedback}_{ij}=(r_i,c_j)∈E (17)"

    The feedback edge is literally a ground-truth positive label expressed as an input edge. The paper never states that the 10% sampled links are held out from the test set used to compute Precision, Recall, and F1 (Eqs. 1-3). Thus, for every sampled positive pair, HGT-All receives a direct edge between exactly the requirement and code pair whose label it is later scored on; the prediction for those pairs is an echo of the input label, not evidence that multi-strategy integration improves TLR. This inflates the reported F1 gains over HGNNLink.

  2. fitted input called prediction [Section 4.3.2, Additional_Information 2 (Gemini prompt)]
    "If user feedback exists for the current requirement–code pair, the following statement is added: Additional_Information 2 User feedback indicates label is 0 or 1."

    In the Gemini-All prompt, the ground-truth label is inserted verbatim as auxiliary input for the pair being judged. Since no split excludes feedback pairs from evaluation, answering 'Yes' to such prompts is reading the supplied label rather than recovering the link from textual similarity plus auxiliary strategies. Gemini-All's reported recall and F1 therefore incorporate a by-construction label supply for the sampled 10% of positive links, directly contributing to the claimed 8.84% average F1 improvement over HGNNLink.

full rationale

The paper's central empirical contribution is the Difference Ratio and the multi-strategy integration results. The Difference Ratio (Eq. 5) is a direct, non-circular statistic of lexical overlap in true versus false links, and its correlation with RoBERTa F1 is an external empirical check, not a tautology. The code-dependency and fine-grained edges are structural and similarity-based inputs independent of the output label. The comparison baselines (TAROT, GA-XWCoDe, HGNNLink) are standard and do not reduce to the proposed methods. However, the user-feedback strategy is circular in evaluation: Eq. (17) injects sampled ground-truth positives as input edges, and the Gemini prompt states the label directly, with no stated train/test split or holdout of these links. The reported F1 gains are therefore partially forced by label leakage. I score this as partial circularity (6), not total (10), because only 10% of positive links are directly label-supplied and the other strategies have independent content. Table inconsistencies (e.g., HGT-All Albergate F1 is 0.5714 in Table 7 but 0.4914 in Table 9; Seam2/Pig rows are duplicated in Table 9) are additional correctness concerns but are not circularity.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central empirical claims rest on dataset quality assumptions, tool correctness, and two hand-chosen thresholds, 10% user feedback and top-20% fine-grained edges. The HGT and MLP dimensions are additional model choices that are not tuned or ablated. No new physical entities are introduced.

free parameters (3)
  • Fine-grained edge rank threshold = top 20% (0.2)
    Eq. 19 adds a fine-grained edge only if all seven code components rank within the top 20% in VSM similarity to the requirement. No sensitivity analysis is provided, and the threshold controls graph connectivity.
  • User feedback sampling ratio = 10%
    Section 4.3.1 randomly samples 10% of ground-truth links as user feedback. The fraction directly controls how much label information enters the model and is not varied across experiments.
  • HGT and MLP architecture dimensions = 128, 64, 512, 256, 128, 64, 1; two HGT layers
    Section 4.3.1 fixes model widths and depth without reporting tuning or ablation, and the central comparisons depend on these choices.
assumptions (6)
  • domain assumption Ground-truth traceability links in the 12 projects are correct and complete.
    All F1 computations treat the public datasets' link sets as exact labels. Any noise or missing links biases the reported precision and recall.
  • domain assumption Spoon-based static analysis correctly extracts import, extends, and method-call dependencies from Java code.
    Code-dependency edges for HGT and the prompt statements for Gemini rely on this extraction. The paper notes JSP files are not analyzed, which may reduce completeness for iTrust.
  • domain assumption RoBERTa and GraphCodeBERT embeddings provide features sufficient for link prediction.
    Node features are fixed vectors from these pretrained models. No fine-tuning or alternative embeddings are compared.
  • domain assumption Randomly sampled 10% of ground-truth links is a faithful simulation of real user feedback.
    Real user feedback may be biased toward easy or important links. The paper assumes uniform sampling is representative.
  • domain assumption Gemini 2.5 Pro zero-shot outputs are a valid proxy for unsupervised TLR performance.
    The unsupervised claim is based on one commercial API at temperature 1.0. Non-determinism and API version drift are not controlled.
  • standard math Statistical tests are applied to valid paired samples.
    Wilcoxon signed-rank and Spearman permutation tests assume paired observations and exchangeability. No violations are discussed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Natural Language-Programming Language Software Traceability Link Recovery Needs More than Textual Similarity." pith.science (2026). https://pith.science/paper/NMURC5AA

@misc{pith2026250905585,
  author       = {Pith},
  title        = {Pith review of: Natural Language-Programming Language Software Traceability Link Recovery Needs More than Textual Similarity},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NMURC5AA}},
  note         = {Machine review of arXiv:2509.05585}
}
read the original abstract

In the field of software traceability link recovery (TLR), textual similarity has long been regarded as the core criterion. However, in tasks involving natural language and programming language (NL-PL) artifacts, relying solely on textual similarity is limited by their semantic gap. To this end, we conducted a large-scale empirical evaluation across various types of TLR tasks, revealing the limitations of textual similarity in NL-PL scenarios. To address these limitations, we propose an approach that incorporates multiple domain-specific auxiliary strategies, identified through empirical analysis, into two models: the Heterogeneous Graph Transformer (HGT) via edge types and the prompt-based Gemini 2.5 Pro via additional input information. We then evaluated our approach using the widely studied requirements-to-code TLR task, a representative case of NL-PL TLR. Experimental results show that both the multi-strategy HGT and Gemini 2.5 Pro models outperformed their original counterparts without strategy integration. Furthermore, compared to the current state-of-the-art method HGNNLink, the multi-strategy HGT and Gemini 2.5 Pro models achieved average F1-score improvements of 3.68% and 8.84%, respectively, across twelve open-source projects, demonstrating the effectiveness of multi-strategy integration in enhancing overall model performance for the requirements-code TLR task.

Figures

Figures reproduced from arXiv: 2509.05585 by the authors.

Figure 1
Figure 1. Research process with the motivation The above observations raise a critical question: Is relying solely on textual similarity sufficient for NL-PL TLR tasks, or should richer strategies be introduced to achieve more accurate and robust traceability link recovery? To address this question, this study first conducted a systematic review of research in this field over the past five years. We performed a statistical an… view at source ↗
Figure 2
Figure 2. Overview of the literature search process [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Literature search and filtering process tool to remove duplicates (applying the exclusion criterion E4), retaining 699 articles after screening. Based on this, we further reviewed the abstracts, introductions, and methodology sections of the articles, applying the inclusion criterion I2 and the exclusion criterion E5 for screening, ultimately retaining 57 articles. We then evaluated the results sections of these art… view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Illustration of Strategy Relationship Extraction for Code Dependency, Fine-Grained, and User Feedback (using the iTrust [PITH_FULL_IMAGE:figures/full_fig_p021_4.png]
Figure 5
Figure 5. Figure 5: Flowchart of the requirements-to-code traceability link recovery using the HGT model [PITH_FULL_IMAGE:figures/full_fig_p022_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

89 extracted references · 76 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774(2023)

    arXiv 2023

  2. [2]

    Nasir Ali, Zohreh Sharafi, Yann-Gaël Guéhéneuc, and Giuliano Antoniol. 2015. An empirical study on the importance of source code entities for requirements traceability.Empirical Software Engineering20 (2015), 442–478

    2015

  3. [3]

    Syed Juned Ali, Varun Naganathan, and Dominik Bork. 2024. Establishing Traceability Between Natural Language Requirements and Software Artifacts by Combining RAG and LLMs. InProceedings of the 44th International Conference on Conceptual Modeling (ER). Springer, 295–314

    2024

  4. [4]

    Zakarea Alshara, Hamzeh Eyal Salman, Anas Shatnawi, and Abdelhak-Djamel Seriai. 2023. Ml-augmented automation for recovering links between pull-requests and issues on GitHub.IEEE Access11 (2023), 5596–5608

    2023

  5. [5]

    Giuliano Antoniol, Gerardo Canfora, Gerardo Casazza, Andrea De Lucia, and Ettore Merlo. 2002. Recovering traceability links between code and documentation.IEEE Transactions on Software Engineering28, 10 (2002), 970–983

    2002

  6. [6]

    Thazin Win Win Aung, Huan Huo, and Yulei Sui. 2020. A literature review of automatic traceability links recovery for software change impact analysis. InProceedings of the 28th International Conference on Program Comprehension (ICPC). ACM, 14–24

    2020

  7. [7]

    Shuotong Bai, Huaxiao Liu, Enyan Dai, and Lei Liu. 2024. Improving Issue-PR Link Prediction via Knowledge-Aware Heterogeneous Graph Learning. IEEE Transactions on Software Engineering50, 7 (2024), 1901–1920

    2024

  8. [8]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners.Advances in Neural Information Processing Systems33 (2020), 1877–1901

    2020

Show all 89 references
  1. [9]

    Sofia Charalampidou, Apostolos Ampatzoglou, Evangelos Karountzos, and Paris Avgeriou. 2021. Empirical studies on software traceability: A mapping study.Journal of Software: Evolution and Process33, 2 (2021), e2294. Manuscript submitted to ACM 34 Zou et al

    2021

  2. [10]

    Lei Chen, Dandan Wang, Lin Shi, and Qing Wang. 2021. A self-enhanced automatic traceability link recovery via structure knowledge mining for small-scale labeled data. InProceedings of the 45th IEEE Annual Computers, Software, and Applications Conference (COMPSAC). IEEE, 904–913

    2021

  3. [11]

    Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 785–794

    2016

  4. [12]

    Jane Cleland-Huang, Orlena CZ Gotel, Jane Huffman Hayes, Patrick Mäder, and Andrea Zisman. 2014. Software traceability: trends and future directions. InProceedings of the 36th IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE)...

    2014

  5. [13]

    Peng Dai, Li Yang, Yawen Wang, Dahai Jin, and Yunzhan Gong. 2023. Constructing traceability links between software requirements and source code based on neural networks.Mathematics11, 2 (2023), 315–339

    2023

  6. [14]

    Yang Deng, Bangchao Wang, Qiang Zhu, Junping Liu, Jiewen Kuang, and Xingfu Li. 2024. MTLink: Adaptive multi-task learning based pre-trained language model for traceability link recovery between issues and commits.Journal of King Saud University-Computer and Information Science...

    2024

  7. [15]

    Yang Deng, Bangchao Wang, Zhiyuan Zou, and Luyao Ye. 2024. PromptLink: Multi-template prompt learning with adversarial training for issue-commit link recovery. InProceedings of the 18th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM). ...

    2024

  8. [16]

    Liming Dong, He Zhang, Wei Liu, Zhiluo Weng, and Hongyu Kuang. 2022. Semi-supervised pre-processing for learning-based traceability framework on real-world software projects. InProceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foun...

    2022

  9. [17]

    Tian-bao Du, Guo-hua Shen, Zhi-qiu Huang, Yao-shen Yu, and De-xiang Wu. 2020. Automatic traceability link recovery via active learning.Frontiers of Information Technology & Electronic Engineering21, 8 (2020), 1217–1225

    2020

  10. [18]

    Alexander Egyed, Florian Graf, and Paul Grünbacher. 2010. Effort and quality of recovering requirements-to-code traces: Two exploratory experiments. InProceedings of the 18th IEEE International Requirements Engineering Conference (RE). IEEE, 221–230

    2010

  11. [19]

    Dominik Fuchß, Tobias Hey, Jan Keim, Haoyu Liu, Niklas Ewald, Tobias Thirolf, and Anne Koziolek. 2025. LiSSA: Toward Generic Traceability Link Recovery through Retrieval-Augmented Generation. InProceedings of the 47th IEEE/ACM International Conference on Software Engineering (...

    2025

  12. [20]

    Hui Gao, Hongyu Kuang, Wesley KG Assunção, Christoph Mayr-Dorn, Guoping Rong, He Zhang, Xiaoxing Ma, and Alexander Egyed. 2024. TRIAD: Automated Traceability Recovery based on Biterm-enhanced Deduction of Transitive Links among Artifacts. InProceedings of the 46th IEEE/ACM Int...

    2024

  13. [21]

    Hui Gao, Hongyu Kuang, Xiaoxing Ma, Hao Hu, Jian Lü, Patrick Mäder, and Alexander Egyed. 2022. Propagating frugal user feedback through closeness of code dependencies to improve IR-based traceability recovery.Empirical Software Engineering27, 2 (2022), 41

    2022

  14. [22]

    Hui Gao, Hongyu Kuang, Kexin Sun, Xiaoxing Ma, Alexander Egyed, Patrick Mäder, Guoping Rong, Dong Shao, and He Zhang. 2022. Using consensual biterms from text structures of requirements and code to improve IR-based traceability recovery. InProceedings of the 37th IEEE/ACM Inte...

    2022

  15. [23]

    Orlena CZ Gotel and CW Finkelstein. 1994. An analysis of the requirements traceability problem. InProceedings of the 1st IEEE International Conference on Requirements Engineering (ICRE). IEEE, 94–101

    1994

  16. [24]

    Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). ACM, 855–864

    2016

  17. [25]

    Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al . 2020. Graphcodebert: Pre-training code representations with data flow.arXiv preprint arXiv:2009.08366(2020)

    2020 arXiv

  18. [26]

    Mouna Hammoudi, Christoph Mayr-Dorn, Atif Mashkoor, and Alexander Egyed. 2021. Tracerefiner: An automated technique for refining coarse-grained requirement-to-class traces. InProceedings of the 28th Asia-Pacific Software Engineering Conference (APSEC). IEEE, 12–21

    2021

  19. [27]

    Shoaib Hassan, Qianmu Li, Khursheed Aurangzeb, Affan Yasin, Javed Ali Khan, and Muhammad Shahid Anwar. 2024. A systematic mapping to investigate the application of machine learning techniques in requirement engineering activities.CAAI Transactions on Intelligence Technology9, ...

    2024

  20. [28]

    Jameleddine Hassine. 2024. An llm-based approach to recover traceability links between security requirements and goal models. InProceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering (EASE). ACM, 643–651

    2024

  21. [29]

    Tobias Hey, Fei Chen, Sebastian Weigelt, and Walter F Tichy. 2021. Improving traceability link recovery using fine-grained requirements-to-code relations. InProceedings of the 37th IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 12–22

    2021

  22. [30]

    John H Holland. 1992. Genetic algorithms.Scientific American267, 1 (1992), 66–73

    1992

  23. [31]

    Ziniu Hu, Yuxiao Dong, Kuansan Wang, and Yizhou Sun. 2020. Heterogeneous graph transformer. InProceedings of the 29th World Wide Web Conference (WWW). ACM, 2704–2710

    2020

  24. [32]

    Jan Keim, Sophie Corallo, Dominik Fuchß, Tobias Hey, Tobias Telge, and Anne Koziolek. 2024. Recovering Trace Links Between Software Documentation And Code. InProceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE). ACM, 1–13

    2024

  25. [33]

    Wiem Khlif, Dhikra Kchaou, and Nadia Bouassida. 2022. A complete traceability methodology between UML diagrams and source code based on enriched use case textual description.Informatica46, 1 (2022), 27–47. Manuscript submitted to ACM NL–PL Software Traceability Link Recovery N...

    2022

  26. [34]

    Jinpeng Lan, Lina Gong, Jingxuan Zhang, and Haoxiang Zhang. 2023. BTLink: automatic link recovery between issues and commits based on pre-trained BERT model.Empirical Software Engineering28, 4 (2023), 103

    2023

  27. [35]

    Tong Li, Shiheng Wang, David Lillis, and Zhen Yang. 2020. Combining machine learning and logical reasoning to improve requirements traceability recovery.Applied Sciences10, 20 (2020), 7253–7276

    2020

  28. [36]

    Xingfu Li, Bangchao Wang, Hongyan Wan, Yuanbang Li, Han Jiang, Yang Deng, and Zhiyuan Zou. 2024. MLTracer: An Approach Based on Multi-Layered Gradient Boosting Decision Trees for Requirements Traceability Recovery. InProceedings of the 36th IEEE International Joint Conference ...

    2024

  29. [37]

    Jinfeng Lin, Yalin Liu, Qingkai Zeng, Meng Jiang, and Jane Cleland-Huang. 2021. Traceability transformed: Generating more accurate links with pre-trained bert models. InProcedings of the 43rd IEEE/ACM International Conference on Software Engineering (ICSE). IEEE, 324–335

    2021

  30. [38]

    Yalin Liu, Jinfeng Lin, and Jane Cleland-Huang. 2020. Traceability support for multi-lingual software projects. InProceedings of the 17th International Conference on Mining Software Repositories (MSR). ACM, 443–454

    2020

  31. [39]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692(2019)

    2019 arXiv

  32. [40]

    Anas Mahmoud, Nan Niu, and Songhua Xu. 2012. A semantic relatedness approach for traceability link recovery. InProceedings of the 20th IEEE International Conference on Program Comprehension (ICPC). IEEE, 183–192

    2012

  33. [41]

    Ali Majidzadeh, Mehrdad Ashtiani, and Morteza Zakeri-Nasrabadi. 2024. Multi-type requirements traceability prediction by code data augmentation and fine-tuning MS-CodeBERT.Computer Standards & Interfaces90 (2024), 103850

    2024

  34. [42]

    Ana C Marcén, Raúl Lapeña, Oscar Pastor, and Carlos Cetina. 2020. Traceability link recovery between requirements and models using an evolutionary algorithm guided by a learning to rank algorithm: Train control and management case.Journal of Systems and Software163 (2020), 110519

    2020

  35. [43]

    Pooya Rostami Mazrae, Maliheh Izadi, and Abbas Heydarnoori. 2021. Automated recovery of issue-commit links leveraging both textual and non-textual data. InProceedings of the 37th IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 263–273

    2021

  36. [44]

    Kevin Moran, David N Palacio, Carlos Bernal-Cárdenas, Daniel McCrystal, Denys Poshyvanyk, Chris Shenefiel, and Jeff Johnson. 2020. Improving the effectiveness of traceability link recovery using hierarchical bayesian networks. InProceedings of the 42nd ACM/IEEE International C...

    2020

  37. [45]

    David Mosquera, Marcela Ruiz, Oscar Pastor, and Jürgen Spielberger. 2024. Understanding the Landscape of Software Modelling Assistants for MDSE tools: A Systematic Mapping.Information and Software Technology173 (2024), 107492

    2024

  38. [46]

    Julia Mucha, Andreas Kaufmann, and Dirk Riehle. 2024. A systematic literature review of pre-requirements specification traceability.Requirements Engineering29 (2024), 1–23

    2024

  39. [47]

    Miguel Angel Olivero, Antonia Bertolino, Francisco José Dominguez-Mayo, María José Escalona, and Ilaria Matteucci. 2024. A systematic mapping study on security for systems of systems.International Journal of Information Security23, 2 (2024), 787–817

    2024

  40. [48]

    Rocco Oliveto, Malcom Gethers, Denys Poshyvanyk, and Andrea De Lucia. 2010. On the equivalence of information retrieval methods for automated traceability link recovery. InProceedings of the 18th IEEE International Conference on Program Comprehension (ICPC). IEEE, 68–71

    2010

  41. [49]

    Sofia Ouhbi, Ali Idri, José Luis Fernández-Alemán, and Ambrosio Toval. 2015. Requirements engineering education: a systematic mapping study. Requirements Engineering20 (2015), 119–138

    2015

  42. [50]

    Zaki Pauzi and Andrea Capiluppi. 2023. Applications of natural language processing in software traceability: A systematic mapping study.Journal of Systems and Software198 (2023), 111616

    2023

  43. [51]

    Tao Peng, Kun She, Yimin Shen, Xiangliang Xu, and Yue Yu. 2023. Enhancing Traceability Link Recovery with Fine-Grained Query Expansion Analysis.Information14, 5 (2023), 270

    2023

  44. [52]

    Kai Petersen, Sairam Vakkalanka, and Ludwik Kuzniarz. 2015. Guidelines for conducting systematic mapping studies in software engineering: An update.Information and Software Technology64 (2015), 1–18

    2015

  45. [53]

    Francisco AC Pinheiro. 2004. Requirements traceability.Perspectives on Software Requirements753 (2004), 91–113

    2004

  46. [54]

    Michael Rath, Jacob Rendall, Jin LC Guo, Jane Cleland-Huang, and Patrick Mäder. 2018. Traceability in the wild: automatically augmenting incomplete trace links. InProceedings of the 40th International Conference on Software Engineering (ICSE). ACM, 834–845

    2018

  47. [55]

    Alberto D Rodriguez, Jane Cleland-Huang, and Davide Falessi. 2021. Leveraging intermediate artifacts to improve automated trace link retrieval. In Proceedings of the 37th IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 81–92

    2021

  48. [56]

    Danissa V Rodriguez and Doris L Carver. 2020. An IR-based artificial bee colony approach for traceability link recovery. InProceedings of the 32nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 1145–1153

    2020

  49. [57]

    Michael Schlichtkrull, Thomas N Kipf, Peter Bloem, Rianne Van Den Berg, Ivan Titov, and Max Welling. 2018. Modeling relational data with graph convolutional networks. InProceedings of the 15th International Conference on Semantic Web (ESWC). Springer, 593–607

    2018

  50. [58]

    Aaron Schlutter and Andreas Vogelsang. 2020. Trace link recovery using semantic relation graphs and spreading activation. InProceedings of the 28th IEEE International Requirements Engineering Conference (RE). IEEE, 20–31

    2020

  51. [59]

    Guohua Shen, Haijuan Wang, Zhiqiu Huang, YaoShen Yu, and Kai Chen. 2021. Supporting requirements to code traceability creation by code comments.International Journal of Software Engineering and Knowledge Engineering31, 08 (2021), 1099–1118

    2021

  52. [60]

    Kexin Sun, Yiding Ren, Hongyu Kuang, Hui Gao, Xiaoxing Ma, Guoping Rong, Dong Shao, and He Zhang. 2024. AVIATE: Exploiting Translation Variants of Artifacts to Improve IR-based Traceability Recovery in Bilingual Software Projects. InProceedings of the 39th IEEE/ACM Internation...

    2024

  53. [61]

    Weifeng Sun, Zhenting Guo, Meng Yan, Zhongxin Liu, Yan Lei, and Hongyu Zhang. 2024. Method-Level Test-to-Code Traceability Link Construction by Semantic Correlation Learning.IEEE Transactions on Software Engineering50, 10 (2024), 2656–2676

    2024

  54. [62]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805(2023)

    2023 arXiv

  55. [63]

    Jiahao Tian, Li Zhang, and Xiaoli Lian. 2023. A cross-level requirement trace link update model based on bidirectional encoder representations from transformers.Mathematics11, 3 (2023), 623

    2023

  56. [64]

    Wouter van Oosten, Randell Rasiman, Fabiano Dalpiaz, and Toine Hurkmans. 2023. On the effectiveness of automated tracing from model changes to project issues.Information and Software Technology160 (2023), 107226

    2023

  57. [65]

    Javier Verón, Raúl Lapeña, Carlos Cetina, Óscar Pastor, and Francisca Pérez. 2024. Improving Requirement Traceability by Leveraging Video Game Simulations in Search-Based Software Engineering. InProceedings of the 36th International Conference on Advanced Information Systems E...

    2024

  58. [66]

    Bangchao Wang, Yang Deng, Ruiqi Luo, Peng Liang, and Tingting Bi. 2025. MPLinker: Multi-template Prompt-tuning with adversarial training for issue-commit Link recovery.Journal of Systems and Software223 (2025), 112351

    2025

  59. [67]

    Bangchao Wang, Yang Deng, Hongyan Wan, and Xingfu Li. 2023. DF4RT: Deep forest for requirements traceability recovery between use cases and source code. InProceedings of the 26th IEEE International Conference on Systems, Man, and Cybernetics (SMC). IEEE, 617–622

    2023

  60. [68]

    Bangchao Wang, Rong Peng, Yuanbang Li, Han Lai, and Zhuo Wang. 2018. Requirements traceability technologies and technology transfer decision support: A systematic review.Journal of Systems and Software146 (2018), 59–79

    2018

  61. [69]

    Bangchao Wang, Rong Peng, Zhuo Wang, Xiaomin Wang, and Yuanbang Li. 2020. An automated hybrid approach for generating requirements trace links.International Journal of Software Engineering and Knowledge Engineering30, 07 (2020), 1005–1048

    2020

  62. [70]

    Bangchao Wang, Heng Wang, Ruiqi Luo, Sen Zhang, and Qiang Zhu. 2022. A systematic mapping study of information retrieval approaches applied to requirements trace recovery.. InProceedings of the 37th International Conference on Software Engineering and Knowledge Engineering (SE...

    2022

  63. [71]

    Bangchao Wang, Zihan Wang, Hongyan Wan, Xingfu Li, and Yang Deng. 2023. An empirical study on data balancing in machine learning based software traceability methods. InProceedings of the 35th International Joint Conference on Neural Networks (IJCNN). IEEE, 1–8

    2023

  64. [72]

    Bangchao Wang, Zhiyuan Zou, Xuanxuan Liang, Huan Jin, and Peng Liang. 2025. HGNNLink: Recovering Requirements-Code Traceability Link with Text and Dependency-Aware Heterogeneous Graph Neural Networks.Automated Software Engineering32, 2 (2025)

    2025

  65. [73]

    Bangchao Wang, Zhiyuan Zou, Hongyan Wan, Yuanbang Li, Yang Deng, and Xingfu Li. 2024. An empirical study on the state-of-the-art methods for requirement-to-code traceability link recovery.Journal of King Saud University-Computer and Information Sciences36, 6 (2024), 102118

    2024

  66. [74]

    Haijuan Wang, Guohua Shen, Zhiqiu Huang, Yaoshen Yu, and Kai Chen. 2021. Analyzing close relations between target artifacts for improving IR-based requirement traceability recovery.Frontiers of Information Technology & Electronic Engineering22, 7 (2021), 957–968

    2021

  67. [75]

    Shiheng Wang, Tong Li, and Zhen Yang. 2019. Using graph embedding to improve requirements traceability recovery. InProceedings of the 2nd International Conference on Applied Informatics (ICAI). Springer, 533–545

    2019

  68. [76]

    Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S Yu. 2019. Heterogeneous graph attention network. InProceedings of the 28th World Wide Web conference (WWW). ACM, 2022–2032

    2019

  69. [77]

    Robert Watkins and Mark Neal. 1994. Why and how of requirements tracing.Ieee Software11, 4 (1994), 104–106

    1994

  70. [78]

    Robert White and Jens Krinke. 2022. TCTracer: Establishing test-to-code traceability links using dynamic and static techniques.Empirical Software Engineering27, 3 (2022), 67

    2022

  71. [79]

    Claes Wohlin. 2014. Guidelines for snowballing in systematic literature studies and a replication in software engineering. InProceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering (EASE). ACM, 1–10

    2014

  72. [80]

    Ayberk Yasa, Cemhan Kaan Ozaltan, Görkem Ayten, Fatih Kaplama, Uçar Omercan Devran, and Tüzün BM. 2025. Evaluating ReLink for Traceability Link Recovery in Practice. InProceedings of the 32nd IEEE International Conference on Software Analysis, Evolution, and Reengineering (SAN...

    2025

  73. [81]

    Insun Yoo, Hyoseung Park, Seok-Won Lee, and Ki-Yeol Ryu. 2024. Building Traceability Between Functional Requirements and Component Architecture Elements in Embedded Software Using Structured Features.Applied Sciences14, 23 (2024), 10796–10819

    2024

  74. [82]

    Chenyuan Zhang, Yanlin Wang, Zhao Wei, Yong Xu, Juhong Wang, Hui Li, and Rongrong Ji. 2023. EALink: An efficient and accurate pre-trained framework for issue-commit link recovery. InProceedings of the 38th IEEE/ACM International Conference on Automated Software Engineering (AS...

    2023

  75. [83]

    Meng Zhang, Chuanqi Tao, Hongjing Guo, and Zhiqiu Huang. 2021. Recovering semantic traceability between requirements and source code using feature representation techniques. InProceedings of the 21st IEEE International Conference on Software Quality, Reliability and Security (...

    2021

  76. [84]

    Jianfei Zhu, Guanping Xiao, Zheng Zheng, and Yulei Sui. 2022. Enhancing traceability link recovery with unlabeled data. InProceedings of the 33rd IEEE International Symposium on Software Reliability Engineering (ISSRE). IEEE, 446–457

    2022

  77. [85]

    Jianfei Zhu, Guanping Xiao, Zheng Zheng, and Yulei Sui. 2024. Deep semi-supervised learning for recovering traceability links between issues and commits.Journal of Systems and Software216 (2024), 112109

    2024

  78. [86]

    Zhiyuan Zou, Bangchao Wang, Yang Deng, Hongyan Wan, Zhiquan An, and Yukun Cao. 2024. XWCoDe: XGBoost with Weighted Code Dependency for Requirements-to-Code Traceability Link Recovery. InProceedings of the 27th IEEE International Conference on Systems, Man, and Cybernetics Manu...

    2024

  79. [87]

    Zhiyuan Zou, Bangchao Wang, Xinrong Hu, Yang Deng, Hongyan Wan, and Huan Jin. 2024. Enhancing requirements-to-code traceability with GA-XWCoDe: Integrating XGBoost, Node2Vec, and genetic algorithms for improving model performance and stability.Journal of King Saud University-C...

    2024

  80. [88]

    Zhiyuan Zou, Bangchao Wang, Peng Liang, and Huan Jin. 2025. Replication Package the Paper: Natural Language-Programming Language Software Traceability Link Recovery Needs More than Textual Similarity. https://github.com/ZZYG0g0g0/NLPL-TLR

    2025

  81. [89]

    SA num” refers to the number of source artifacts, “TA num

    Zhiyuan Zou, Bangchao Wang, Hongyan Wan, Huan Jin, Xiaoxiao Li, and Yukun Cao. 2024. HANTracer: Leveraging Heterogeneous Graph Attention Network for Large-Scale Requirements-Code Traceability Link Recovery. InProceedings of the 31st Asia-Pacific Software Engineering Conference...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.