Pith. sign in

REVIEW 6 major objections 6 minor 113 references

Assessing Large Language Models in Comprehending and Verifying Concurrent Programs across Memory Models

T0 review · 6 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Five leading LLMs can spot simple concurrency bugs under sequential consistency, but all of them fail to verify even small programs once relaxed memory ordering is allowed.

desk verdict Useful first look at LLMs on relaxed-memory verification, but the TSO claims rest on PSO-only ground truth and the abstract oversells deadlock detection. read the letter →

arxiv 2501.14326 v2 pith:KXVRRQVY submitted 2025-01-24 cs.SE

Ridhi Jain , Rahul Purandare This is my paper · ORCID
classification cs.SE
keywords largelanguagemodelsconcurrentprogramsrelaxedmemorytotalstoreorderpartialdataracedetectionprogramverificationlitmustests
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Five leading LLMs (GPT-3.5-turbo, GPT-4, GPT-4o, GPT-4o-mini, and Mistral Large2) were asked to summarize concurrent C/C++ programs, identify data races and deadlocks, and judge whether assertions can fail, first under sequential consistency and then under the relaxed memory models TSO and PSO. The paper's central finding is a split: several models can detect obvious race and deadlock problems in sequentially consistent programs, but none can be relied on to verify even small programs once store reordering is permitted. On 25 ARM litmus tests designed to expose TSO/PSO reorderings, GPT-4 performed best (F1 of 0.80) yet still failed to provide correct causes for assertion failures, and models typically described TSO and PSO correctly in the abstract while misapplying the rules to concrete code. If correct, the result implies that LLMs are not ready as standalone verification tools for relaxed-memory concurrency and should at best complement model checkers.

What carries the argument

The evaluation apparatus is a prompt-benchmark pipeline: five fixed questions (Q1–Q5) appended to stripped, comment-free programs; 25 ARM litmus tests using C11 std::memory_order_relaxed atomics as the TSO/PSO probes; SV-COMP pthread programs as the sequential-consistency probes; Nidhugg and ESBMC model checkers to fix ground truth for assertion outcomes; and human inspection to bin verbose LLM answers into ✓/✗ labels. The load-bearing step is the manual binarization: the paper converts fluent prose into binary correctness judgments, and every accuracy table depends on that conversion being faithful.

What would settle it

Re-run Q5 on the same 25 litmus tests with the same five models but have two independent annotators, blind to model identity, label the responses, and report inter-annotator agreement; if agreement on 'correct reasoning' is low, the 0.72 accuracy claim for GPT-4 is not a stable measurement. Alternatively, give each model a list of all candidate store orderings (SC, TSO, and PSO) and ask it only to choose which ordering falsifies the assertion: if a model that failed the open-ended prompt now picks correctly, the bottleneck is trace generation, not memory-model semantics.

Watch

Extended reading notes

Core claim

The paper claims that current LLMs understand concurrency at the level of textbook knowledge but not at the level of operational verification. Under a sequentially consistent memory model, GPT-4, GPT-4o, and Mistral Large2 effectively identified data races and deadlocks in the SV-COMP pthread benchmarks and often answered correctly whether assertions in those programs always hold. Under TSO and PSO, however, all five models fell short: even the best performer, GPT-4, achieved only 0.72 accuracy and 0.80 F1 on predicting assertion failures in the litmus tests, and every model struggled to produce feasible reordering traces or correct causal explanations, sometimes attributing failures to inter-thread reordering when the actual issue is intra-thread store reordering. The paper takes this as evidence that LLMs lack a reliable internal model of memory-ordering constraints.

Load-bearing premise

The whole evaluation leans on the ground-truth labels produced by the model checkers and on the authors' manual classification of wordy LLM answers; if those labels are wrong or biased, every accuracy figure in the tables changes.

Editorial extensions

If this is right

  • For detecting simple data races and deadlocks under sequential consistency, the better LLMs are already usable as a quick first pass, but their output must be independently checked.
  • For relaxed memory models, no evaluated LLM is reliable enough to replace or even pre-filter a model checker, so verification of TSO/PSO behavior should remain with tools built for that semantics.
  • The benchmark questions and the public dataset provide a reusable testbed for future models; a model that cannot beat GPT-4's 0.80 F1 on these 25 litmus tests is unlikely to generalize to larger relaxed-memory programs.
  • Hybrid designs that let an LLM explain or generate candidate executions while a model checker decides validity are the natural next step suggested by the paper's data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A plausible reading of the failure pattern is that the LLMs memorize textbook statements about TSO and PSO but lack the operational ability to enumerate intra-thread store orderings coherently, since they can state the rule and still fail to produce the interleaving that breaks the assertion.
  • The results suggest a testable boundary: if the same litmus tests were posed as multiple-choice trace questions with concrete candidate interleavings rather than open-ended 'can the assertion fail?' questions, accuracy might rise sharply; the paper's open-ended format may confound trace-generation weakness with memory-model misunderstanding.
  • Because litmus tests are tiny and specifically engineered to expose reordering, the paper's negative result likely understates the gap on realistic concurrent code, where the LLM would also have to identify which accesses are shared; conversely, fine-tuning on litmus-test-style traces may be a cheap way to close part of the gap.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper evaluates five LLMs (GPT-3.5-turbo, GPT-4, GPT-4o, GPT-4o-mini, and Mistral Large2) on their ability to comprehend and verify small concurrent C/C++ programs. It uses 25 ARM litmus tests and SV-COMP pthread benchmarks, with ground truth derived from ESBMC and Nidhugg. The study asks whether LLMs can summarize concurrent programs, detect data races and deadlocks, verify assertions under sequential consistency, and predict assertion outcomes under relaxed memory models (TSO and PSO). The main reported finding is that models perform reasonably on SC-related tasks but fail to verify small programs under relaxed memory models.

Significance. If the claims were properly scoped, this would be a useful empirical contribution: the negative result that state-of-the-art LLMs cannot reliably verify even small litmus tests under relaxed memory models is actionable for researchers building LLM-based verification assistants. The paper ships an anonymous artifact containing prompts, datasets, and LLM responses, and uses established verification tools for ground truth. However, several load-bearing claims currently overstate what the measurements support: the TSO-specific conclusion is not supported by the PSO-only ground truth, the deadlock claim contradicts Table 4, and the relaxed-memory tests contain at least one program that does not use the stated relaxed atomics. These issues are fixable and the core PSO negative result appears robust, but the paper needs a major revision.

major comments (6)
  1. [Section 4.6.3, Table 5] The ground truth for all 25 litmus tests is reported as generated by Nidhugg with only the --pso option, while the paper's stated scope (abstract, Section 3.1) includes both TSO and PSO. Since PSO is strictly weaker than TSO, the set of behaviors in the Ground Truth column of Table 5 is not a valid proxy for TSO; for example, the 2+2W pattern in Listing 2 depends on store-store reordering, which TSO does not allow. The reported accuracy and F1 values therefore do not support any conclusion about TSO. The authors should either produce separate --tso ground truth columns or explicitly restrict the RQ4 conclusion to PSO.
  2. [Section 4.6.4 vs Section 4.7, Listing 1] The paper states that all read/write operations in the litmus tests are tagged with std::memory_order_relaxed, but Listing 1 (CoRW) calls atomic_store(&x, 1), atomic_load(&x), and atomic_store(&a, 1) without a memory-order argument, which uses the default memory_order_seq_cst. As a result, the LLM is asked to reason about a program whose semantics do not match the intended relaxed-memory ground truth for at least one test in Table 5. The code should be corrected to use memory_order_relaxed, or the ground truth for CoRW should be recomputed under the actual program semantics.
  3. [Abstract, Table 4 (Q3)] The abstract claims that GPT-4, GPT-4o, and Mistral Large2 'effectively identify data races and deadlocks' when assessed under a sequentially consistent memory model, but Table 4 shows that every evaluated model has precision 0 on deadlock detection in the SV-COMP set (e.g., GPT-4: TP=0, FP=2, TN=32; GPT-4o: TP=0, FP=20, TN=14). None of the models correctly identified a single deadlock, and all positive deadlock predictions were false positives. This sentence should be revised, and the related discussion in Sections 4.6.2 and 5.1.2 should be aligned with these numbers.
  4. [Section 4.6.3 (RQ3)] The research question promises a comparison to traditional static/dynamic analysis tools 'in terms of both accuracy and efficiency,' but the section only uses ESBMC and Nidhugg to generate ground truth. There is no comparison of LLM accuracy against tool accuracy on the benchmarks, and no efficiency measurements (e.g., runtime or resource usage) are reported. The section should either be reframed as describing ground-truth generation, or actual comparison data should be added.
  5. [Section 3.4, Table 5] The manual labeling procedure is described only at a high level, and the caption of Table 5 introduces ad-hoc conventions for '-' (treated as false), '◦' (partial information), and similar symbols in Table 4 ('LL' treated as true, '!' as false). No detailed rubric or inter-rater agreement is reported, yet the binary labels determine every accuracy, precision, recall, and F1 value in the paper. A reproducible annotation protocol should be provided in the artifact, or the claims should be correspondingly qualified.
  6. [Section 4.3, Tables 3–5] Each LLM was queried once with default temperature settings, and no repeated runs or statistical tests are reported. Consequently, differences such as GPT-4's F1=0.80 versus Mistral's F1=0.21 in Table 5 could be within sampling noise. The authors should report variance over repeated prompts or add appropriate significance testing for the ranking claims.
minor comments (6)
  1. [Figure 1] The figure contains the typo 'partal store order' (should be 'partial').
  2. [Section 4.6.4] The text refers to 'Research Question 5 (RQ5)', but the paper only defines RQ1–RQ4; this should be RQ4.
  3. [Table 4 caption] The caption says 'We consider LL as ✓ and ! as ✗ while computing TP, FP, TN and TN'; the final 'TN' should presumably be 'FN'.
  4. [Section 4.6.2] The phrase 'Tables 3 ans 4' should read 'Tables 3 and 4'.
  5. [Section 3.2 (1)] The sentence contains a duplicated phrase: 'LLMs show promise in LLMs show promise in understanding and summarizing code.'
  6. [Table 1] The header contains 'GPT-40-mini' (should be 'GPT-4o-mini'), and the footnote markers '2' after 'Modality' and 'Output Tokens' are not defined.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the study is an empirical benchmark with external ground truth, and no prediction reduces to a fitted input or self-citation.

full rationale

This paper performs a measurement task: it prompts LLMs on SV-COMP pthread benchmarks and 25 litmus tests, manually labels responses, and compares labels against ground truth generated by ESBMC and Nidhugg. There is no derivation chain of the kind that can be circular: no parameter is fitted to the data and then renamed as a prediction; the Q5 correctness-under-relaxed-memory-model scores are computed against Nidhugg's --pso results (Section 4.6.3) plus manual verification, which is external evidence rather than an output of the LLM or of the paper's own construction. The authors' self-citations (e.g., [45], BIRD) appear only in related work and are not load-bearing for the measured accuracies. Concerns that the ground truth conflates TSO/PSO or that C11 relaxed-atom programs are not identical to hardware PSO are validity threats to the benchmark, not cases where the conclusion is equivalent to the input by definition; under the stated rules those concerns belong to correctness risk, not circularity. The manual labeling in Section 3.4 is subjective but is a measurement protocol, not a self-referential prediction. Accordingly, no circular step is identified and the circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper's central claims rest on the correctness of external model checkers as ground truth, on the validity of mapping C11 relaxed atomics to TSO and PSO, and on the reliability of the authors' manual labeling. No new entities are introduced.

free parameters (3)
  • Litmus test subset (25 tests)
    A hand-picked subset of ARM litmus tests; selection criteria are not stated. The accuracy numbers depend on which tests are included.
  • SV-COMP pthread subset (34 tests)
    A hand-picked subset of SV-COMP benchmarks with no inclusion criteria given. The results depend on this composition.
  • Manual labeling rubric
    Authors converted free-form LLM outputs to binary correct or incorrect labels without publishing a rubric or inter-rater agreement. This choice determines all TP, FP, TN, and FN counts.
assumptions (4)
  • domain assumption Nidhugg and ESBMC correctly implement SC, TSO, and PSO semantics for the test programs.
    Used to derive ground truth in Section 4.6.3; if these tools mis-model the C11 atomics or the ARM litmus tests, the ground truth is wrong.
  • domain assumption Equating C11 memory_order_relaxed programs with TSO and PSO hardware behavior is valid.
    The paper treats relaxed atomics as if they run under TSO and PSO in Sections 3.1 and 4.6.4; this correspondence is not justified, since PSO permits store-store reordering that relaxed atomics may not directly map to on all implementations.
  • domain assumption Manual response labeling is accurate and unbiased.
    Section 3.4 describes manual inspection without a rubric, blind labeling, or inter-rater reliability. All quantitative results rely on these labels.
  • domain assumption A single LLM run at default temperature represents model capability.
    Section 4.3 uses default temperature and does not report repeated sampling; LLM output is stochastic, so single-run results may not be stable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Assessing Large Language Models in Comprehending and Verifying Concurrent Programs across Memory Models." pith.science (2026). https://pith.science/paper/KXVRRQVY

@misc{pith2026250114326,
  author       = {Pith},
  title        = {Pith review of: Assessing Large Language Models in Comprehending and Verifying Concurrent Programs across Memory Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KXVRRQVY}},
  note         = {Machine review of arXiv:2501.14326}
}
read the original abstract

As concurrent programming becomes increasingly prevalent, effectively identifying and addressing concurrency issues such as data races and deadlocks is critical. This study evaluates the performance of several leading large language models (LLMs), including GPT-3.5-turbo, GPT-4, GPT-4o, GPT-4o-mini, and Mistral-AI's Large2, in understanding and analyzing concurrency issues within software programs. Given that relaxed memory models, such as Total Store Order (TSO) and Partial Store Order (PSO), are widely implemented and adapted in modern systems, supported even by commodity architectures like ARM and x86, our evaluation focuses not only on sequentially consistent memory models but also on these relaxed memory models. Specifically, we assess two main aspects: the models' capacity to detect concurrency problems under a sequentially consistent memory model and their ability to verify the correctness conditions of concurrent programs across both sequentially consistent and relaxed memory models. To do this, we leverage SV-COMP's pthread tests and 25 ARM Litmus tests designed to evaluate Total Store Order (TSO) and Partial Store Order (PSO) memory models. The experimental results reveal that GPT-4, GPT-4o, and Mistral-AI's Large2 demonstrate a robust understanding of concurrency issues, effectively identifying data races and deadlocks when assessed under a sequentially consistent memory model. However, despite its superior performance, all selected LLMs face significant challenges verifying program correctness under relaxed memory models. These LLMs exhibit limitations in accurately capturing memory ordering constraints, and their current capabilities fall short in verifying even small programs in these complex scenarios.

Figures

Figures reproduced from arXiv: 2501.14326 by the authors.

Figure 1
Figure 1. An overview of our approach. Feature GPT-4o GPT-4 GPT-3.5-turbo GPT-40-mini Mistral-Large2 Initial Release Date May 13, 2024 March 14, 2023 Nov, 2022 July 24, 2024 July 24, 2024 Modality2 Text and Images Text and Images Text and Images Text and Images Text and Images Output Tokens2 4096 8192 4096 16384 8192 Parameters [14, 82, 106] 70B1 1.8T3 175B 8B 123B Cost per 1M to￾ken [74, 84] Input: $5 Input: $30 Input: $3 In… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

113 extracted references · 53 canonical work pages

  1. [1]

    A., Aronis, S., Atig, M

    Abdulla, P. A., Aronis, S., Atig, M. F., Jonsson, B., Leonardsson, C., and Sagonas, K. Stateless model checking for tso and pso. Acta Informatica 54 (2017), 789–818

    2017

  2. [2]

    Litmus: Running tests against hardware

    Alglave, J., Maranget, L., Sarkar, S., and Sewell, P. Litmus: Running tests against hardware. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (2011), Springer, pp. 41–44

    2011

  3. [3]

    Can llms patch security issues? arXiv preprint arXiv:2312.00024 (2023)

    Alrashedy, K., and Aljasser, A. Can llms patch security issues? arXiv preprint arXiv:2312.00024 (2023)

    arXiv 2023

  4. [4]

    In 2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) (2024), IEEE, pp

    Ashiwal, V., Finster, S., and Dawoud, A.Llm-based vulnerability sourcing from unstructured data. In 2024 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) (2024), IEEE, pp. 634–641

    2024

  5. [5]

    arXiv preprint arXiv:2108.07732 (2021)

    Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al.Program synthesis with large language models. arXiv preprint arXiv:2108.07732 (2021)

    arXiv 2021

  6. [6]

    Proceedings of Machine Learning and Systems 3 (2021), 181–193

    Baghdadi, R., Merouani, M., Leghettas, M.-H., Abdous, K., Arbaoui, T., Benatchba, K., et al.A deep learning based cost model for automatic code optimization. Proceedings of Machine Learning and Systems 3 (2021), 181–193

    2021

  7. [7]

    ACM SIGPLAN Notices 51, 1 (2016), 539–554

    Bao, W., Krishnamoorthy, S., Pouchet, L.-N., Rastello, F., and Sadayappan, P.Polycheck: Dynamic verification of iteration space transformations on affine programs. ACM SIGPLAN Notices 51, 1 (2016), 539–554

    2016

  8. [8]

    arXiv preprint arXiv:2402.13291 (2024)

    Berabi, B., Gronskiy, A., Raychev, V., Sivanrupan, G., Chibotaru, V., and Vechev, M.Deepcode ai fix: Fixing security vulnerabilities with large language models. arXiv preprint arXiv:2402.13291 (2024)

    arXiv 2024

Show all 113 references
  1. [9]

    Competition on software verification and witness validation: Sv-comp 2023

    Beyer, D. Competition on software verification and witness validation: Sv-comp 2023. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (2023), Springer, pp. 495–522

    2023

  2. [10]

    Cpa-symexec: efficient symbolic execution in cpachecker

    Beyer, D., and Lemberger, T. Cpa-symexec: efficient symbolic execution in cpachecker. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (2018), pp. 900–903

    2018

  3. [11]

    W., and Sergey, I

    Blackshear, S., Gorogiannis, N., O’Hearn, P. W., and Sergey, I. Racerd: compositional static race detection. Proceedings of the ACM on Programming Languages 2 , OOPSLA (2018), 1–28

    2018

  4. [12]

    In Automated Technology for Verification and Analysis: 13th International Symposium, ATV A 2015, Shanghai, China, October 12-15, 2015, Proceedings 13 (2015), Springer, pp

    Bliudze, S., Cimatti, A., Jaber, M., Mover, S., Roveri, M., Saab, W., and W ang, Q.Formal verification of infinite-state bip models. In Automated Technology for Verification and Analysis: 13th International Symposium, ATV A 2015, Shanghai, China, October 12-15, 2015, Proceedin...

    2015

  5. [13]

    Repairagent: An autonomous, llm-based agent for program repair

    Bouzenia, I., Devanbu, P., and Pradel, M. Repairagent: An autonomous, llm-based agent for program repair. arXiv preprint arXiv:2403.17134 (2024)

    2024 arXiv

  6. [14]

    Brown, T. B. Language models are few-shot learners. arXiv preprint ArXiv:2005.14165 (2020)

    2020 arXiv

  7. [15]

    Brick: Path enumeration based bounded reachability checking of c program (competition contribution)

    Bu, L., Xie, Z., Lyu, L., Li, Y., Guo, X., Zhao, J., and Li, X. Brick: Path enumeration based bounded reachability checking of c program (competition contribution). In International Conference on Tools and Algorithms for the Construction and Analysis of Systems (2022), Springe...

    2022

  8. [16]

    Klee symbolic execution engine in 2019

    Cadar, C., and Nowack, M. Klee symbolic execution engine in 2019. International Journal on Software Tools for Technology Transfer 23 (2021), 867–870

    2021

  9. [17]

    Codegraph: Enhancing graph reasoning of llms with code

    Cai, Q., Wang, Z., Diao, S., Kwok, J., and Song, Y. Codegraph: Enhancing graph reasoning of llms with code. arXiv preprint arXiv:2408.13863 (2024)

    2024 arXiv

  10. [18]

    Bgnn4vd: Constructing bidirectional graph neural-network for vulnerability detection

    Cao, S., Sun, X., Bo, L., Wei, Y., and Li, B. Bgnn4vd: Constructing bidirectional graph neural-network for vulnerability detection. Information and Software Technology 136 (2021), 106576

    2021

  11. [19]

    Carpenter, G. A. Neural network models for pattern recognition and associative memory. Neural networks 2, 4 (1989), 243–257

    1989

  12. [20]

    A., and Cordeiro, L

    Charalambous, Y., Tihanyi, N., Jain, R., Sun, Y., Ferrag, M. A., and Cordeiro, L. C. A new era in software security: Towards self-healing software via large language models and formal verification. arXiv preprint arXiv:2305.14752 (2023)

    2023 arXiv

  13. [21]

    Data race detection using large language models

    Chen, L., Ding, X., Emani, M., Vanderbruggen, T., Lin, P.-H., and Liao, C. Data race detection using large language models. In Proceedings of the SC’23 Workshops of The International Conference on High Performance Computing, Network, Storage, and Analysis (2023), pp. 215–223

    2023

  14. [22]

    Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021)

    2021 arXiv

  15. [23]

    ACM Transactions on Software Engineering and Methodology (TOSEM) 30 , 3 (2021), 1–33

    Cheng, X., W ang, H., Hua, J., Xu, G., and Sui, Y.Deepwukong: Statically detecting software vulnerabilities using deep graph neural network. ACM Transactions on Software Engineering and Methodology (TOSEM) 30 , 3 (2021), 1–33

    2021

  16. [24]

    Hippodrome: Data race repair using static analysis summaries

    Costea, A., Tiwari, A., Chianasta, S., Roychoudhury, A., and Sergey, I. Hippodrome: Data race repair using static analysis summaries. ACM Transactions on Software Engineering and Methodology 32 , 2 (2023), 1–33. , Vol. 1, No. 1, Article . Publication date: September 2025. Asse...

    2023

  17. [25]

    Cui, H., Xie, M., Su, T., Zhang, C., and Tan, S. H. An empirical study of false negatives and positives of static code analyzers from the perspective of historical issues, 2024

    2024

  18. [26]

    V., Ben-Nun, T., Hoefler, T., and Leather, H.Programl: Graph-based deep learning for program optimization and analysis

    Cummins, C., Fisches, Z. V., Ben-Nun, T., Hoefler, T., and Leather, H.Programl: Graph-based deep learning for program optimization and analysis. arXiv preprint arXiv:2003.10536 (2020)

    2020 arXiv

  19. [27]

    A rule-based verification strategy for array manipulating programs

    De Angelis, E., Fioravanti, F., Pettorossi, A., and Proietti, M. A rule-based verification strategy for array manipulating programs. Fundamenta Informaticae 140, 3-4 (2015), 329–355

    2015

  20. [28]

    Satcheck: Sat-directed stateless model checking for sc and tso

    Demsky, B., and Lam, P. Satcheck: Sat-directed stateless model checking for sc and tso. ACM SIGPLAN Notices 50, 10 (2015), 20–36

    2015

  21. [29]

    S., Peng, H., Y ang, C., and Zhang, L.Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models

    Deng, Y., Xia, C. S., Peng, H., Y ang, C., and Zhang, L.Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT international symposium on software testing and analysis (2023), pp. 423–435

    2023

  22. [30]

    Vulngpt: Enhancing source code vulnerability detection using autogpt and adaptive supervision strategies

    Eberhardt, G., and Milánkovich, Á. Vulngpt: Enhancing source code vulnerability detection using autogpt and adaptive supervision strategies. In 2024 20th International Conference on Distributed Computing in Smart Systems and the Internet of Things (DCOSS-IoT) (2024), IEEE, pp. 450–454

    2024

  23. [31]

    De-hallucinator: Iterative grounding for llm-based code completion

    Eghbali, A., and Pradel, M. De-hallucinator: Iterative grounding for llm-based code completion. arXiv preprint arXiv:2401.01701 (2024)

    2024 arXiv

  24. [32]

    Scientific Programming 2019, 1 (2019), 6230953

    Fan, G., Diao, X., Yu, H., Y ang, K., and Chen, L.Software defect prediction via attention-based recurrent neural network. Scientific Programming 2019, 1 (2019), 6230953

    2019

  25. [33]

    Fan, Z., Gao, X., Roychoudhury, A., and Tan, S. H. Automated repair of programs from large language models. arXiv preprint arXiv:2205.10583 (2022)

    2022 arXiv

  26. [34]

    A., Battah, A., Tihanyi, N., Debbah, M., Lestable, T., and Cordeiro, L

    Ferrag, M. A., Battah, A., Tihanyi, N., Debbah, M., Lestable, T., and Cordeiro, L. C. Securefalcon: The next cyber reasoning system for cyber security. arXiv preprint arXiv:2307.06616 (2023)

    2023 arXiv

  27. [35]

    R., Monteiro, F

    Gadelha, M. R., Monteiro, F. R., Morse, J., Cordeiro, L. C., Fischer, B., and Nicole, D. A. Esbmc 5.0: an industrial-strength c model checker. In ACM/IEEE ASE (2018), pp. 888–891

    2018

  28. [36]

    Search-based llms for code optimization

    Gao, S., Gao, C., Gu, W., and Lyu, M. Search-based llms for code optimization. arXiv preprint arXiv:2408.12159 (2024)

    2024 arXiv

  29. [37]

    Dynamic data race detection for openmp programs

    Gu, Y., and Mellor-Crummey, J. Dynamic data race detection for openmp programs. In SC18: International Conference for High Performance Computing, Networking, Storage and Analysis (2018), IEEE, pp. 767–778

    2018

  30. [38]

    Hasanzad, E., and Babamir, S. M. Prediction of deadlocks in concurrent programs using neural network

  31. [39]

    Rule-based runtime verification revisited

    Havelund, K. Rule-based runtime verification revisited. International Journal on Software Tools for Technology Transfer 17 (2015), 143–170

    2015

  32. [40]

    J., and Devanbu, P

    Hellendoorn, V. J., and Devanbu, P. Are deep neural networks the best choice for modeling source code? In Proceedings of the 2017 11th Joint meeting on foundations of software engineering (2017), pp. 763–773

    2017

  33. [41]

    Cigar: Cost-efficient program repair with llms

    Hidvégi, D., Etemadi, K., Bobadilla, S., and Monperrus, M. Cigar: Cost-efficient program repair with llms. arXiv preprint arXiv:2402.06598 (2024)

    2024 arXiv

  34. [42]

    Degpt: Optimizing decompiler output with llm

    Hu, P., Liang, R., and Chen, K. Degpt: Optimizing decompiler output with llm. In Proceedings 2024 Network and Distributed System Security Symposium (2024). https://api. semanticscholar. org/CorpusID (2024), vol. 267622140

    2024

  35. [43]

    Deep-autocoder: Learning to complete code precisely with induced code tokens

    Hu, X., Men, R., Li, G., and Jin, Z. Deep-autocoder: Learning to complete code precisely with induced code tokens. In 2019 IEEE 43rd Annual Computer Software and Applications Conference (COMPSAC) (2019), vol. 1, IEEE, pp. 159–168

    2019

  36. [44]

    In Proceedings of the 44th International Conference on Software Engineering (2022), pp

    Jain, N., V aidyanath, S., Iyer, A., Natarajan, N., Parthasarathy, S., Rajamani, S., and Sharma, R.Jigsaw: Large language models meet program synthesis. In Proceedings of the 44th International Conference on Software Engineering (2022), pp. 1219–1231

    2022

  37. [45]

    Bird: Race detection in software binaries under relaxed memory models

    Jain, R., Purandare, R., and Sharma, S. Bird: Race detection in software binaries under relaxed memory models. ACM Transactions on Software Engineering and Methodology (TOSEM) 31 , 4 (2022), 1–29

    2022

  38. [46]

    Inferfix: End-to-end program repair with llms

    Jin, M., Shahriar, S., Tufano, M., Shi, X., Lu, S., Sundaresan, N., and Svyatkovskiy, A. Inferfix: End-to-end program repair with llms. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (2023)...

    2023

  39. [47]

    Why don’t software developers use static analysis tools to find bugs? In 35th International Conference on Software Engineering (ICSE) (2013), IEEE, pp

    Johnson, B., Song, Y., Murphy-Hill, E., and Bowdidge, R. Why don’t software developers use static analysis tools to find bugs? In 35th International Conference on Software Engineering (ICSE) (2013), IEEE, pp. 672–681

    2013

  40. [48]

    C., Gulwani, S., Le, V., Verbruggen, G., and Radiček, I.Repair is nearly generation: Multilingual program repair with llms

    Joshi, H., Sanchez, J. C., Gulwani, S., Le, V., Verbruggen, G., and Radiček, I.Repair is nearly generation: Multilingual program repair with llms. In Proceedings of the AAAI Conference on Artificial Intelligence (2023), vol. 37, pp. 5131–5140

    2023

  41. [49]

    Verasco: a formally verified C static analyzer

    Jourdan, J.-H. Verasco: a formally verified C static analyzer . PhD thesis, Universite Paris Diderot-Paris VII, 2016

    2016

  42. [50]

    (security) assertions by large language models

    Kande, R., Pearce, H., Tan, B., Dolan-Gavitt, B., Thakur, S., Karri, R., and Rajendran, J. (security) assertions by large language models. IEEE Transactions on Information Forensics and Security (2024)

    2024

  43. [51]

    Kester, D., Mwebesa, M., and Bradbury, J. S. How good is static analysis at finding concurrency bugs? In 10th IEEE Working Conference on Source Code Analysis and Manipulation (2010), IEEE, pp. 115–124

    2010

  44. [52]

    Dynamic symbolic verification of mpi programs

    Khanna, D., Sharma, S., Rodríguez, C., and Purandare, R. Dynamic symbolic verification of mpi programs. In Formal Methods: 22nd International Symposium, FM 2018, Held as Part of the Federated Logic Conference, FloC 2018, Oxford, UK, July 15-17, 2018, Proceedings 22 (2018), Spr...

    2018

  45. [53]

    In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (2020), pp

    Kokologiannakis, M., and Vafeiadis, V.Hmc: Model checking for hardware memory models. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (2020), pp. 1157–1171

    2020

  46. [54]

    D., Savarese, S., and Hoi, S

    Le, H., W ang, Y., Gotmare, A. D., Savarese, S., and Hoi, S. C. H.Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems 35 (2022), 21314–21328

    2022

  47. [55]

    Improved code summarization via a graph neural network

    LeClair, A., Haqe, S., Wu, L., and McMillan, C. Improved code summarization via a graph neural network. In Proceedings of the 28th international conference on program comprehension (2020), pp. 184–195

    2020

  48. [56]

    Leinonen, J., Hellas, A., Sarsa, S., Reeves, B., Denny, P., Prather, J., and Becker, B. A. Using large language models to enhance programming error messages. In Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 (2023), , Vol. 1, No. 1, Article ...

    2023

  49. [57]

    Enhancing static analysis for practical bug detection: An llm-integrated approach

    Li, H., Hao, Y., Zhai, Y., and Qian, Z. Enhancing static analysis for practical bug detection: An llm-integrated approach. Proceedings of the ACM on Programming Languages 8 , OOPSLA1 (2024), 474–499

    2024

  50. [58]

    Li, J., He, P., Zhu, J., and Lyu, M. R. Software defect prediction via convolutional neural network. In 2017 IEEE international conference on software quality, reliability and security (QRS) (2017), IEEE, pp. 318–328

    2017

  51. [59]

    Llm-assisted static analysis for detecting security vulnerabilities

    Li, Z., Dutta, S., and Naik, M. Llm-assisted static analysis for detecting security vulnerabilities. arXiv preprint arXiv:2405.17238 (2024)

    2024 arXiv

  52. [60]

    D., and Jabbarvand, R

    Liu, C., Zhang, S. D., and Jabbarvand, R. Codemind: A framework to challenge large language models for code reasoning. arXiv preprint arXiv:2402.09664 (2024)

    2024 arXiv

  53. [61]

    Stall+: Boosting llm-based repository-level code completion with static analysis

    Liu, J., Chen, Y., Liu, M., Peng, X., and Lou, Y. Stall+: Boosting llm-based repository-level code completion with static analysis. arXiv preprint arXiv:2406.10018 (2024)

    2024 arXiv

  54. [62]

    Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

    Liu, J., and et al. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. In NeurIPS (2024)

    2024

  55. [63]

    S., W ang, Y., and Zhang, L.Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

    Liu, J., Xia, C. S., W ang, Y., and Zhang, L.Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. arXiv preprint arXiv:2305.01210 (2023)

    2023 arXiv

  56. [64]

    S., W ang, Y., and Zhang, L.Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation

    Liu, J., Xia, C. S., W ang, Y., and Zhang, L.Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2024)

    2024

  57. [65]

    Deepstate: selecting test suites to enhance the robustness of recurrent neural networks

    Liu, Z., Feng, Y., Yin, Y., and Chen, Z. Deepstate: selecting test suites to enhance the robustness of recurrent neural networks. In Proceedings of the 44th International Conference on Software Engineering (2022), pp. 598–609

    2022

  58. [66]

    Grace: Empowering llm-based software vulnerability detection with graph structure and in-context learning

    Lu, G., Ju, X., Chen, X., Pei, W., and Cai, Z. Grace: Empowering llm-based software vulnerability detection with graph structure and in-context learning. Journal of Systems and Software 212 (2024), 112031

    2024

  59. [67]

    I., TehraniJamsaz, A., Phan, H

    Mahmud, Q. I., TehraniJamsaz, A., Phan, H. D., Ahmed, N. K., and Jannesari, A. Autoparllm: Gnn-guided automatic code parallelization using large language models. arXiv preprint arXiv:2310.04047 (2023)

    2023 arXiv

  60. [68]

    S.Dynamic verification for hybrid concurrent programming models

    Mutlu, E., Gajinov, V., Cristal, A., Tasiran, S., and Unsal, O. S.Dynamic verification for hybrid concurrent programming models. In Runtime Verification: 5th International Conference, RV 2014, Toronto, ON, Canada, September 22-25, 2014. Proceedings 5 (2014), Springer, pp. 156–161

    2014

  61. [69]

    Nicholas, G., and Bhatia, A.Lost in translation: large language models in non-english content analysis.arXiv preprint arXiv:2306.07377 (2023)

    2023 arXiv

  62. [70]

    H., Xie, Z., Rajaram, A., and Bhatele, A

    Nichols, D., Davis, J. H., Xie, Z., Rajaram, A., and Bhatele, A. Can large language models write parallel code? In Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing (2024), pp. 281–294

    2024

  63. [71]

    Modeling parallel programs using large language models

    Nichols, D., Marathe, A., Menon, H., Gamblin, T., and Bhatele, A. Modeling parallel programs using large language models. arXiv preprint arXiv:2306.17281 (2023)

    2023 arXiv

  64. [72]

    Cdschecker: checking concurrent data structures written with c/c++ atomics

    Norris, B., and Demsky, B. Cdschecker: checking concurrent data structures written with c/c++ atomics. In Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications (2013), pp. 131–150

    2013

  65. [73]

    Fuzzing busybox: Leveraging llm and crash reuse for embedded bug unearthing

    Oliinyk, Y., Scott, M., Tsang, R., Fang, C., Homayoun, H., et al. Fuzzing busybox: Leveraging llm and crash reuse for embedded bug unearthing. arXiv preprint arXiv:2403.03897 (2024)

    2024 arXiv

  66. [74]

    OpenAI. Pricing. https://openai.com/api/pricing, 2024. [Accessed 06-June-2024]

    2024

  67. [75]

    Vulnerabilities introduced by llms through code suggestions

    Panichella, S. Vulnerabilities introduced by llms through code suggestions. In Large Language Models in Cybersecurity: Threats, Exposure and Mitigation. Springer Nature Switzerland Cham, 2024, pp. 87–97

    2024

  68. [76]

    Examining zero-shot vulnerability repair with large language models

    Pearce, H., Tan, B., Ahmad, B., Karri, R., and Dolan-Gavitt, B. Examining zero-shot vulnerability repair with large language models. In SP (2022), IEEE, pp. 1–18

    2022

  69. [77]

    Examining zero-shot vulnerability repair with large language models

    Pearce, H., Tan, B., Ahmad, B., Karri, R., and Dolan-Gavitt, B. Examining zero-shot vulnerability repair with large language models. In 2023 IEEE Symposium on Security and Privacy (SP) (2023), IEEE, pp. 2339–2356

    2023

  70. [78]

    V., Le Nguyen, M., and Bui, L

    Phan, A. V., Le Nguyen, M., and Bui, L. T. Convolutional neural networks over control flow graphs for software defect prediction. In 2017 IEEE 29th International Conference on Tools with Artificial Intelligence (ICTAI) (2017), IEEE, pp. 45–52

    2017

  71. [79]

    Qin, F., Zheng, Z., Sui, Y., Gong, S., Shi, Z., and Trivedi, K. S. Cross-project concurrency bug prediction using domain-adversarial neural network. Journal of Systems and Software 214 (2024), 112077

    2024

  72. [80]

    A., and Engler, D.{Under-Constrained} symbolic execution: Correctness checking for real code

    Ramos, D. A., and Engler, D.{Under-Constrained} symbolic execution: Correctness checking for real code. In 24th USENIX Security Symposium (2015), pp. 49–64

    2015

  73. [81]

    A., Kadosh, T., Krien, N., Capota, M., Tamir, G., Willke, T

    Schneider, N., Hasabnis, N., Vo, V. A., Kadosh, T., Krien, N., Capota, M., Tamir, G., Willke, T. L., Ahmed, N., Pinter, Y., et al. Mpirigen: Mpi code generation through domain-specific language models. In Proceedings of the 2024 Workshop on AI For Systems , pp. 1–6

    2024

  74. [82]

    Gpt-4 architecture, datasets, costs and more leaked

    Schreiner, M. Gpt-4 architecture, datasets, costs and more leaked. https://the-decoder.com/gpt-4-architecture-datasets-costs-and- more-leaked, July 2023. [Accessed 07-June-2024]

    2023

  75. [83]

    Dlpaper2code: Auto-generation of code from deep learning research papers

    Sethi, A., Sankaran, A., Panwar, N., Khare, S., and Mani, S. Dlpaper2code: Auto-generation of code from deep learning research papers. In Proceedings of the AAAI Conference on Artificial Intelligence (2018), vol. 32

    2018

  76. [84]

    Openai api pricing 2024: Understanding gpt-3 pricing in-depth

    Sheremetov, D., and Bitkina, A. Openai api pricing 2024: Understanding gpt-3 pricing in-depth. https://onix-systems.com/blog/how- much-does-it-cost-to-use-gpt-models, June 2023. [Accessed 07-June-2024]

    2024

  77. [85]

    Dynamic verification of c11 concurrency over multi copy atomics

    Singh, S., Sharma, D., and Sharma, S. Dynamic verification of c11 concurrency over multi copy atomics. In 2021 International Symposium on Theoretical Aspects of Software Engineering (TASE) (2021), IEEE, pp. 39–46

    2021

  78. [86]

    U., Ramamurthy, K

    Sosa, R. U., Ramamurthy, K. N., Chang, M., and Singh, M. Reasoning about concepts with llms: Inconsistencies abound. In First Conference on Language Modeling

  79. [87]

    Pythia: Ai-assisted code completion system

    Svyatkovskiy, A., Zhao, Y., Fu, S., and Sundaresan, N. Pythia: Ai-assisted code completion system. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (2019), pp. 2727–2735

    2019

  80. [88]

    Llm4decompile: Decompiling binary code with large language models

    Tan, H., Luo, Q., Li, J., and Zhang, Y. Llm4decompile: Decompiling binary code with large language models. arXiv preprint , Vol. 1, No. 1, Article . Publication date: September 2025. Assessing Large Language Models in Comprehending and Verifying Concurrent Programs across Memo...

    2024 arXiv

  81. [89]

    Deeprace: finding data race bugs via deep learning

    Tehrani, A., Khaleel, M., Akbari, R., and Jannesari, A. Deeprace: finding data race bugs via deep learning. arXiv preprint arXiv:1907.07110 (2019)

    2019 arXiv

  82. [90]

    A., Zhang, M., Mathur, U., and Viswanathan, M

    Thokair, M. A., Zhang, M., Mathur, U., and Viswanathan, M. Dynamic race detection with o (1) samples. Proceedings of the ACM on Programming Languages 7 , POPL (2023), 1308–1337

    2023

  83. [91]

    O., Tang, X., Cheung, S.-C., Klein, J., and Bissyandé, T

    Tian, H., Lu, W., Li, T. O., Tang, X., Cheung, S.-C., Klein, J., and Bissyandé, T. F.Is chatgpt the ultimate programming assistant–how far is it? arXiv preprint arXiv:2304.11938 (2023)

    2023 arXiv

  84. [92]

    Improving llm code generation with grammar augmentation

    Ugare, S., Suresh, T., Kang, H., Misailovic, S., and Singh, G. Improving llm code generation with grammar augmentation. arXiv preprint arXiv:2403.01632 (2024)

    2024 arXiv

  85. [93]

    Llms cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework, and benchmarks

    Ullah, S., Han, M., Pujar, S., Pearce, H., Coskun, A., and Stringhini, G. Llms cannot reliably identify and reason about security vulnerabilities (yet?): A comprehensive evaluation, framework, and benchmarks. In IEEE Symposium on Security and Privacy (2024)

    2024

  86. [94]

    A., Godoy, W

    V alero-Lara, P., Huante, A., Lail, M. A., Godoy, W. F., Teranishi, K., Balaprakash, P., and Vetter, J. S.Comparing llama-2 and gpt-3 llms for hpc kernels generation. arXiv preprint arXiv:2309.07103 (2023)

    2023 arXiv

  87. [95]

    IEEE Transactions on Services Computing 15, 3 (2020), 1649–1663

    W ang, B., Xu, L., Y an, M., Liu, C., and Liu, L.Multi-dimension convolutional neural network for bug localization. IEEE Transactions on Services Computing 15, 3 (2020), 1649–1663

    2020

  88. [96]

    IEEE Transactions on Software Engineering (2024)

    W ang, J., Huang, Y., Chen, C., Liu, Z., W ang, S., and W ang, Q.Software testing with large language models: Survey, landscape, and vision. IEEE Transactions on Software Engineering (2024)

    2024

  89. [97]

    N., W ang, S., Li, Y., Zhang, J., and Y adavally, A.Deepvd: Toward class-separation features for neural network vulnerability detection

    W ang, W., Nguyen, T. N., W ang, S., Li, Y., Zhang, J., and Y adavally, A.Deepvd: Toward class-separation features for neural network vulnerability detection. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (2023), IEEE, pp. 2249–2261

    2023

  90. [98]

    arXiv preprint arXiv:2203.05132 (2022)

    W ang, X., W ang, Y., W an, Y., Mi, F., Li, Y., Zhou, P., Liu, J., Wu, H., Jiang, X., and Liu, Q.Compilable neural code generation with compiler feedback. arXiv preprint arXiv:2203.05132 (2022)

    2022 arXiv

  91. [99]

    Complex & Intelligent Systems 9 , 4 (2023), 3835–3855

    W ang, Z., Tong, W., Li, P., Ye, G., Chen, H., Gong, X., and Tang, Z.Bugpre: an intelligent software version-to-version bug prediction system using graph convolutional neural networks. Complex & Intelligent Systems 9 , 4 (2023), 3835–3855

    2023

  92. [100]

    S., and Zhang, L

    Wei, Y., Xia, C. S., and Zhang, L. Copiloting the copilots: Fusing large language models with completion engines for automated program repair. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering...

    2023

  93. [101]

    Enchanting program specification synthesis by large language models using static analysis and program verification

    Wen, C., Cao, J., Su, J., Xu, Z., Qin, S., He, M., Li, H., Cheung, S.-C., and Tian, C. Enchanting program specification synthesis by large language models using static analysis and program verification. In International Conference on Computer Aided Verification (2024), Springe...

    2024

  94. [102]

    A., Arqint, L., Clochard, M., Oortwijn, W., Pereira, J

    Wolf, F. A., Arqint, L., Clochard, M., Oortwijn, W., Pereira, J. C., and Müller, P. Gobra: Modular specification and verification of go programs. In International Conference on Computer Aided Verification (2021), Springer, pp. 367–379

    2021

  95. [103]

    S., Paltenghi, M., Le Tian, J., Pradel, M., and Zhang, L.Fuzz4all: Universal fuzzing with large language models

    Xia, C. S., Paltenghi, M., Le Tian, J., Pradel, M., and Zhang, L.Fuzz4all: Universal fuzzing with large language models. InProceedings of the IEEE/ACM 46th International Conference on Software Engineering (2024), pp. 1–13

    2024

  96. [104]

    S., Wei, Y., and Zhang, L

    Xia, C. S., Wei, Y., and Zhang, L. Automated program repair in the era of large pre-trained language models. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) (2023), IEEE, pp. 1482–1494

    2023

  97. [105]

    Krace: Data race fuzzing for kernel file systems

    Xu, M., Kashyap, S., Zhao, H., and Kim, T. Krace: Data race fuzzing for kernel file systems. In 2020 IEEE Symposium on Security and Privacy (SP) (2020), IEEE, pp. 1643–1660

    2020

  98. [106]

    A survey on multilingual large language models: Corpora, alignment, and bias

    Xu, Y., Hu, L., Zhao, J., Qiu, Z., Ye, Y., and Gu, H. A survey on multilingual large language models: Corpora, alignment, and bias. arXiv preprint arXiv:2404.00929 (2024)

    2024 arXiv

  99. [107]

    Z., Le Goues, C., Martins, R., and Hellendoorn, V.Large language models for test-free fault localization

    Y ang, A. Z., Le Goues, C., Martins, R., and Hellendoorn, V.Large language models for test-free fault localization. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (2024), pp. 1–12

    2024

  100. [108]

    Kernelgpt: Enhanced kernel fuzzing via large language models

    Yang, C., Zhao, Z., and Zhang, L. Kernelgpt: Enhanced kernel fuzzing via large language models. arXiv preprint arXiv:2401.00563 (2023)

    2023 arXiv

  101. [109]

    In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (2023), pp

    Zhang, J., W ang, X., Zhang, H., Sun, H., Liu, X., Hu, C., and Liu, Y.Detecting condition-related bugs with control flow graph neural network. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (2023), pp. 1370–1382

    2023

  102. [110]

    On the effectiveness of large language models for github workflows

    Zhang, X., Muralee, S., Cherupattamoolayil, S., and Machiry, A. On the effectiveness of large language models for github workflows. arXiv preprint arXiv:2403.12446 (2024)

    2024 arXiv

  103. [111]

    Detecting code comment inconsistencies using llm and program analysis

    Zhang, Y. Detecting code comment inconsistencies using llm and program analysis. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering (2024), pp. 683–685

    2024

  104. [112]

    Neural Computing and Applications 34, 18 (2022), 15441–15455

    Zhang, Y., Y an, J., Qiao, L., and Gao, H.A novel approach of data race detection based on cnn-bilstm hybrid neural network. Neural Computing and Applications 34, 18 (2022), 15441–15455

    2022

  105. [113]

    Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks

    Zhou, Y., Liu, S., Siow, J., Du, X., and Liu, Y. Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks. Advances in neural information processing systems 32 (2019). , Vol. 1, No. 1, Article . Publication date: Sept...

    2019

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.