Pith. sign in

REVIEW 4 major objections 5 minor 61 references

PDLogger: Automated Logging Framework for Practical Software Development

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PDLogger claims automated logging works by giving LLMs precise program context.

desk verdict Solid multi-log logging pipeline with real novelty, but the evaluation's ground truth is public-project logs that the backbone LLMs likely memorized, and the 'consistently outperforms' claim is contradicted by its own Table 5. read the letter →

arxiv 2507.19951 v1 pith:JAMK7SG5 submitted 2025-07-26 cs.SE

Shengcheng Duan , Yihua Xu , Sheng Zhang , Shen Wang , Yue Duan This is my paper · ORCID
classification cs.SE
keywords automatedlogginglogstatementgenerationlargelanguagemodelsprogramslicingpositionpredictionmulti-logsoftwarediagnosabilitycodeblockanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PDLogger sets out to show that automated logging can work in realistic code, where a single method often needs several log statements. Its thesis is that the bottleneck is not the language model itself but the context it receives: the paper argues that block-type-aware prompts, backward program slices carrying control and data dependencies, and an expanded variable list are what let an LLM produce complete logs that match developer intent. The paper reports that on 3,113 log statements from Hadoop and ActiveMQ this three-phase pipeline beats the Multi-SCLogger baseline by 139.0% in position precision, 69.2% in F1, 82.3% in level accuracy, 131.8% in variable precision, and 65.7% in BERTScore. A sympathetic reader would take the contribution to be a reusable recipe for turning LLM-based log generation from a single-position toy task into a deployable, multi-log tool.

What carries the argument

The central mechanism is the block-type-based structured prompt combined with backward program slicing. Each method is annotated with the start and end lines of its branch, try-catch, loop, and method-definition blocks; the LLM is queried per block type with heuristic rules such as avoiding logs that remain unchanged across loop iterations and evaluating each catch block individually. From each predicted log position, a backward slice of at most seven hops traces the control and data dependencies that explain the statement's execution condition, giving the LLM the root-cause context needed to write a diagnostic message. A function-aware variable extension adds member, inherited, interface, lambda, and statically imported functions to the candidate variable list, and a refinement stage applies level correction plus five language-agnostic deduplication rules before insertion.

What would settle it

Probe whether the 3,113 ground-truth log lines appear in the training corpora of o3-mini, DeepSeek, and LLaMA-3 via membership-inference testing; if they do, run PDLogger on logs committed after each model's training cutoff and check whether the reported F1 advantage over Multi-SCLogger persists.

Watch

Extended reading notes

Core claim

PDLogger's central claim is that complete log statements for real methods can be generated end-to-end if the LLM is pointed at the right code. The paper identifies three deficiencies in prior work — no support for multiple logs per method, missing inter-procedural semantic dependencies, and too narrow a variable scope — and answers all three with a three-phase pipeline. Phase one predicts log positions by annotating branch, try-catch, loop, and method-definition blocks and querying the LLM with block-specific heuristic prompts; phase two runs backward program slicing from each predicted position and augments the candidate variable set with member, inherited, interface, lambda, and statically imported functions; phase three corrects log levels and removes redundant logs using five deduplication rules. On 3,113 logs from Hadoop 3.4.1 and ActiveMQ 5.18.7, PDLogger reports large gains over Multi-SCLogger in every component, and the ablation study ties each gain to a specific phase, e.g., removing block-type prompts nearly quadruples false-positive positions and removing semantic-dependency expansion drops BERTScore by 15.1%.

Load-bearing premise

The evaluation assumes that the developer-written logs in Hadoop 3.4.1 and ActiveMQ 5.18.7 are the complete and correct ground truth, and that the backbone LLMs have not memorized these exact public log statements during pretraining.

Editorial extensions

If this is right

  • If PDLogger's results hold, automated logging can be extended from inserting one log per method to inserting several, matching the 1.77 logs-per-method average the paper reports from LogBench-O.
  • The block-type prompt templates and the deletion-priority deduplication rules are presented as model-agnostic and language-agnostic, so other log-generation systems could adopt them without reimplementing PDLogger.
  • Because the framework's gains persist across o3-mini, LLaMA-3-70B, and DeepSeek-Chat, the design choices of slicing, variable expansion, and refinement, rather than a particular LLM, carry the improvement.
  • PDLogger is intended to be usable on log-free projects: it accepts source code and outputs an augmented project with logs inserted, a capability the paper says prior end-to-end approaches lack.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper evaluates against developer-written logs as ground truth; an implication the paper leaves implicit is that the same pipeline could serve as a logging-consistency checker, flagging places where an existing log omits a variable or level that semantic context suggests.
  • A natural extension is to apply the three phases to non-Java languages; since the load-bearing parts are AST block extraction and program slicing, the main porting cost is static-analysis tooling rather than the LLM prompts.
  • The five deduplication rules encode an implicit theory of when logs are redundant, such as keeping else-branch logs over then-branch logs and dropping start logs post-dominated by end logs; testing these rules as standalone heuristics against human judgment could give them independent value beyond the full pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper presents PDLogger, an LLM-based framework for generating complete logging statements (position, level, message, and variables) in both single- and multi-log scenarios. The framework operates in three phases: block-type-aware structured prompts for position prediction, backward program slicing with function-aware variable extraction for log generation, and level refinement with deduplication for pruning false positives. The authors evaluate PDLogger on 3,113 logs from Apache Hadoop 3.4.1 and Apache ActiveMQ 5.18.7, reporting large improvements over a multi-log variant of SCLogger (position precision +139%, F1 +69.2%, level accuracy +82.3%, variable precision +131.8%, BERTScore +65.7%), along with an ablation study and a generalizability study across three LLMs. The implementation is open source.

Significance. If the reported results hold, PDLogger would represent a meaningful step toward practical automated logging, being explicitly designed for methods containing multiple logs and incorporating inter-procedural semantic dependencies that prior approaches largely ignore. The open-source release and evaluation across three LLM backbones are strengths, and the ablation study gives some evidence for the contribution of each component. However, the central quantitative claims are currently threatened by a likely training-data contamination risk, by the use of the same projects for both design decisions and evaluation, and by incomplete baseline coverage. The significance is therefore conditional on addressing these threats in a revision.

major comments (4)
  1. [Section 5, Threats to Validity; Tables 2, 3, 5] The Threats to Validity paragraph discusses data leakage only for proprietary code, but the evaluation uses logs from two widely used public repositories, Hadoop 3.4.1 and ActiveMQ 5.18.7, which are plausible constituents of the pretraining corpora for o3-mini, DeepSeek-V3, and LLaMA-3-70B. Because the task asks the model to regenerate exact developer-written logs, the models may have memorized these statements and their positions. All headline metrics in Tables 2, 3, and 5 are measured against these logs, so the reported gains may partly reflect retrieval of memorized ground truth rather than generative skill. The comparison against Multi-SCLogger does not fully control for this: PDLogger's prompts reconstruct more of the original method layout, potentially making memorized logs easier to elicit. I recommend adding a contamination analysis, e.g., holding out logs from projects or versions whose timestamps postdate the LLMs' training cutoffs, or testing whether the models can reproduce the exact log from a minimal prompt without the surrounding code, and reporting the results.
  2. [Section 2.3 and Table 1; Section 4.1] The design of PDLogger is partly motivated by empirical observations on the same projects used for evaluation. The 13% external-function variable observation in Section 2.3 is derived from Apache Hadoop, and the block-type heuristic rules in Table 1 are said to be based on 'empirical analysis of log placement principles' without stating whether that analysis was performed on held-out data. Given that the evaluation set in Section 4.1 also consists of Hadoop and ActiveMQ, the framework may be tuned to the test distribution, inflating the reported gains. For instance, the function-aware extension is designed specifically to address the 13% Hadoop observation, and Table 3 shows a large variable-precision gain on that same project. To support the generalizability claim, the authors should evaluate on at least one held-out project not used in any design decision, or clearly separate design-time analyses from evaluation-time data.
  3. [Section 4.1, Tables 2 and 3] The baseline selection is incomplete for the claims made. The multi-log evaluation compares only against Multi-SCLogger, a variant constructed by the authors, while recent end-to-end logging systems such as UniLog [49], FastLog [48], and LoGFiLM [57] are discussed in related work but never included in the experiments, nor adapted to the multi-log setting. The single-log evaluation includes LANCE and LANCE 2.0 but omits these newer systems as well. Without a broader comparison, the statement that PDLogger 'vastly outperforms state-of-the-art log generation techniques' is not fully supported. The authors should either include these baselines (adapted to the multi-log setting where appropriate) or provide a concrete justification for why they are not comparable.
  4. [Section 4.2 and Table 2] The evaluation lacks statistical rigor. The dataset contains 3,113 logs from two projects, but no confidence intervals, per-project breakdowns, or significance tests are reported. Because logging practices vary considerably across projects, the headline improvements could be driven primarily by one of the two projects. Moreover, the sampling procedure is described as 'randomly select 3,113 log statements,' but it is not clear whether the unit of sampling was methods or logs, which affects the interpretation of the F1 scores. I recommend reporting per-project results, the numbers of sampled methods and logs per project, and at least basic dispersion measures such as bootstrap confidence intervals for the main metrics.
minor comments (5)
  1. [Section 4.1] Clarify the sampling unit (methods vs. logs) and provide per-project counts for Hadoop and ActiveMQ, since the current description is ambiguous.
  2. [Tables 3 and 5] In Table 3, the column header 'PrecisionRecall F1' for the variable group is missing spaces; fix the formatting. In Table 5, the model name 'Deepseek-chat' is used inconsistently with 'DeepSeek-V3' in Section 4.1; standardize the naming.
  3. [Section 3.3] The notation for function sets uses uppercase V (e.g., Vi, Vd, Vl, Vs) for functions, which conflicts with the earlier use of V for variable sets; consider renaming the function sets (e.g., Fi, Fd, Fl, Fs) to avoid confusion.
  4. [Section 5] The limitation statement says 'Evaluation uses mostly Java projects,' but the evaluation actually uses only Java projects; change 'mostly' to 'all'.
  5. [Abstract and Section 1] The claim of being the 'first end-to-end log generation technique expressly designed for practical, multi-log scenarios' is strong; consider softening it or explicitly contrasting with FastLog and other recent end-to-end systems to avoid overstatement.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PDLogger's headline results are measured against external developer-written ground truth; its same-project motivation observations are a validity concern, not a circular derivation.

full rationale

PDLogger is an empirical systems paper rather than a formal derivation, so the pattern of 'predicted quantity defined in terms of its inputs' is absent. The three design phases (block-type prompts, backward-slicing plus function-aware variable extraction, and refinement) are engineering choices that are validated by experiments on developer-written logs in Hadoop 3.4.1 and ActiveMQ 5.18.7, with a baseline comparison and an ablation study; none of the headline metrics is an identity by construction. The motivation study does draw on the same project later used for evaluation: Section 1 reports that '13% of the logs utilize non-member functions or function expressions as variables' from '100 randomly sampled logs from Apache Hadoop', and Table 1's rules are said to be 'derived through ... our empirical analysis of log placement principles'. This creates a risk that design choices are tuned to the test distribution, weakening external validity, but it is not circular in the required sense: no parameter is fitted to the test-set ground-truth labels, and the evaluation is against externally written logs rather than against the motivating observations themselves. The Threats to Validity section (Section 5) addresses data leakage only for 'proprietary code' and omits contamination risk for the public benchmark logs; that is a serious evaluation threat but a contamination/correctness concern, not a circularity. There are no load-bearing self-citations: SCLogger, LANCE, and the block-type study are independent prior works. Accordingly, the honest finding is no significant circularity (score 0).

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

This is an empirical systems paper with no mathematical derivation. The hand-chosen parameters (7-hop slice limit, 1-hop context bundle, position tolerance) and domain assumptions about LLM behavior and ground-truth quality carry the load. No new theoretical entities are postulated.

free parameters (3)
  • Backward slice hop limit = 7
    Chosen to fit LLM input length; controls how much dependency context is available for message generation.
  • Code Context Bundle hop count = 1
    Used in deduplication to select surrounding methods for redundancy checks; hand-chosen scope.
  • Position accuracy tolerance = ±1 line within same block
    Evaluation threshold inherited from LANCE; the position accuracy scores in Table 2 depend on this choice.
assumptions (5)
  • domain assumption LLMs can generate high-quality logging statements when given code, block context, and dependency information.
    The entire method relies on this capability; no independent validation is provided outside the evaluation logs.
  • domain assumption Existing developer-written logs in Hadoop 3.4.1 and ActiveMQ 5.18.7 are a valid ground truth for correct logging.
    Dataset construction uses these logs as labels without cross-validation or expert review; see Section 4.1.
  • ad hoc to paper Backward slicing with at most seven hops captures the semantic dependencies needed for good log messages.
    The seven-hop limit is a practical constraint, not an empirically established bound; introduced in Section 3.3.
  • ad hoc to paper The four block-type heuristic rules improve position prediction precision.
    Rules in Table 1 are derived from prior work and empirical observation, but are not separately validated.
  • domain assumption Hadoop and ActiveMQ are representative of real-world Java logging practices.
    Only two projects are used; the authors themselves note cross-language generality is uncertain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PDLogger: Automated Logging Framework for Practical Software Development." pith.science (2026). https://pith.science/paper/JAMK7SG5

@misc{pith2026250719951,
  author       = {Pith},
  title        = {Pith review of: PDLogger: Automated Logging Framework for Practical Software Development},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JAMK7SG5}},
  note         = {Machine review of arXiv:2507.19951}
}
read the original abstract

Logging is indispensable for maintaining the reliability and diagnosability of modern software, yet developers still struggle to decide where and how to log effectively. Existing automated logging techniques focus on isolated sub-tasks - predicting a single log position, level, or message - and therefore cannot produce complete, high-quality log statements that reflect real-world practice in which multiple logs often appear inside one method. They also neglect deeper semantic dependencies among methods and consider only a narrow set of candidate variables, leading to superficial or incomplete logs. In this paper, we present PDLogger, the first end-to-end log generation technique expressly designed for practical, multi-log scenarios. PDLogger operates in three phases. (1) Log position prediction: block-type-aware structured prompts guide a large language model (LLM) to suggest candidate positions across all control-flow blocks of a method. (2) Log generation: backward program slicing supplies precise inter-procedural control and data-dependency context, while an expanded variable extractor captures both member and external function expressions; the enriched prompt enables the LLM to emit a full log statement (position, level, message, variables). (3) Log refinement: level correction and context-sensitive deduplication prune false positives and redundant logs. We evaluate PDLogger on 3,113 log statements drawn from two widely used Java projects. Compared with the strongest prior systems, PDLogger improves log-position precision by 139.0 percent, F1 by 69.2 percent, level accuracy by 82.3 percent, variable precision by 131.8 percent, and message quality (BERTScore) by 65.7 percent. The framework consistently performs well with different mainstream LLMs, demonstrating robustness and generality. PDLogger's implementation is available as open source to foster future research and adoption.

Figures

Figures reproduced from arXiv: 2507.19951 by the authors.

Figure 1
Figure 1. Multi-Log Generation Issues We illustrate this limitation with an example method from the ActiveMQ [1] project, shown in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Semantic Dependency Tracking. The red￾highlighted line marks the starting statement of the backward slice. Blue-highlighted lines denote statements reached through data and control dependency tracing [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Overview of PDLogger [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Log Position Prediction Workflow To enhance the precision of position in multi-log prediction while reducing the false positive rate of position, each prompt for a different type of block incorporates a set of heuristic rules, shown [PITH_FULL_IMAGE:figures/full_fig_p…
Figure 6
Figure 6. Figure 6: Log Generation Workflow approach[26] to expanding the variable scope, we further summa￾rize the information of functions that are likely to appear as logging variables. We first adopt the variable sets 𝑣 defined in SCLogger, where 𝑣 ∈ 𝑉𝑝 ∪𝑉𝑚 ∪𝑉𝑐 ∪𝑉𝑠 ∪𝑉𝑖[55]. Here,𝑉𝑝 de…
Figure 7
Figure 7. Figure 7: Log Refinement Workflow and severity of log messages, enabling developers to control log output across different environments, facilitate efficient debugging, and support runtime monitoring and alerting. Based on our in￾vestigation and some prior studies [33, 51], we i…
Figure 8
Figure 8. Figure 8: A Case Study in the Single-log Generation Task [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: A Case Study in the Multi-Log Generation Scenario [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Case Study of the Ablation Study [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 54 canonical work pages

  1. [49]

    Jiaxu Xu, Zhiqing Cui, Yihao Zhao, Xin Zhang, Sheng He, Peng He, Lin Li, Yu Kang, Qiang Lin, Yuan Dang, and Shrinivasan Rajmohan. 2024. Unilog: Auto- matic Logging via LLM and In-Context Learning. In46th IEEE/ACM International Conference on Software Engineering (ICSE). 1–12

    work page 2024

  2. [48]

    Xiaoxue Xie, Zhiyuan Cai, Sheng Chen, and Jing Xuan. 2024. FastLog: An End- to-End Method to Efficiently Generate and Insert Logging Statements. In33rd Conference acronym ’XX, June 03–05, 2018, Woodstock, NY Trovato et al. ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 26–37

    work page 2024

  3. [57]

    Hongyu Zhang, Dasen Yu, Liuzhe Zhang, Guoping Rong, Yiyang Yu, Hong Shen, Hongbo Zhang, Dong Shao, and Haoyu Kuang. 2024. LoGFiLM: Fine-Tuning a Large Language Model for Automated Generation of Log Statements.arXiv preprint arXiv:2412.18835(2024)

    arXiv 2024

  4. [1]

    Apache ActiveMQ: Version 5.18.7

    2025. Apache ActiveMQ: Version 5.18.7. https://activemq.apache.org/ components/classic/download/

    work page 2025

  5. [2]

    Meta AI. 2024. LLaMA 3. https://ai.meta.com/llama/. Accessed July 2025

    work page 2024

  6. [3]

    Abhay Amar and Peter C. Rigby. 2019. Mining Historical Test Logs to Predict Bugs and Localize Faults. In41st IEEE/ACM International Conference on Software Engineering (ICSE). 140–151

    work page 2019

  7. [4]

    Apache. 2023. Apache Log4j 2. https://logging.apache.org/log4j/2.x/

    work page 2023

  8. [5]

    Apache. 2025. Apache Hadoop: Release 3.4.1. https://hadoop.apache.org/releases. html

    work page 2025

Show all 61 references
  1. [6]

    Paul A. Cohen. 2003. Java AST and Compiler Reuse via the Eclipse JDT. In Conference on Object-Oriented Programming, Systems, Languages and Applications (OOPSLA). 123–136

    2003

  2. [7]

    DeepSeek. 2024. DeepSeek-Chat. https://github.com/deepseek-ai/DeepSeek-LLM. Accessed July 2025

    2024

  3. [8]

    Rui Ding, Huailin Zhou, Jian-Guang Lou, Hongyu Zhang, Qiang Lin, Qiang Fu, Dongmei Zhang, and Tao Xie. 2015. Log2: A Cost-Aware Logging Mechanism for Performance Diagnosis. InUSENIX Annual Technical Conference (USENIX ATC 15). 139–150

    2015

  4. [9]

    Yuhang Ding, Benjamin Steenhoek, Kexin Pei, Gail Kaiser, Wei Le, and Baishakhi Ray. 2024. TRACED: Execution-Aware Pre-Training for Source Code. In46th IEEE/ACM International Conference on Software Engineering (ICSE). 1–12

    2024

  5. [10]

    Zhaoning Ding, Ying Tang, Xiaoning Cheng, Heng Li, and Weiyi Shang. 2023. LogEnText-plus: Improving Neural Machine Translation Based Logging Texts Generation with Syntactic Templates.ACM Transactions on Software Engineering and Methodology33, 2 (2023), 1–45

    2023

  6. [11]

    Qiang Fu, Jieming Zhu, Wenlu Hu, Jian-Guang Lou, Rui Ding, Qingwei Lin, Dongmei Zhang, and Tao Xie. 2014. Where Do Developers Log? An Empirical Study on Logging Practices in Industry. InCompanion Proceedings of the 36th International Conference on Software Engineering (ICSE). 24–33

    2014

  7. [12]

    Yuhan Fu, Minrui Yan, Peng He, Chang Liu, Xiaoning Zhang, and Di Yang. 2024. End-to-End Log Statement Generation at Block Level.Journal of Systems and Software216 (2024), 112–146

    2024

  8. [13]

    Saeid Gholamian. 2021. Leveraging Code Clones and Natural Language Process- ing for Log Statement Prediction. In36th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1043–1047

    2021

  9. [14]

    Saeid Gholamian and Patrick A. S. Ward. 2021. What Distributed Systems Say: A Study of Seven Spark Application Logs.arXiv preprint arXiv:2108.08395(2021)

    2021 arXiv

  10. [15]

    Shenghui Gu, He Zhang, Wanggen Liu, and Guoping Rong. 2023. LoGenText-Plus: Improving Neural Machine Translation-based Logging Texts Generation with Syntactic Templates.ACM Transactions on Software Engineering and Methodology (TOSEM)32, 3 (2023), 61:1–61:33

    2023

  11. [16]

    C. Gulcu. 2023. SLF4J: Simple Logging Facade for Java. https://www.slf4j.org

    2023

  12. [17]

    Jiawei He, Mohit Rungta, Daniel Koleczek, Aman Sekhon, Feng-Xiang Wang, and Shafiq Hasan. 2024. Does Prompt Formatting Have Any Impact on LLM Performance?arXiv preprint arXiv:2411.10541(2024)

    2024 arXiv

  13. [18]

    Zhouyang Jia, Shanshan Li, Xiaodong Liu, Xiangke Liao, and Yunhuai Liu. 2018. SMARTLOG: Place error log statement by deep understanding of log intention. In2018 IEEE 25th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 61–71

    2018

  14. [19]

    Zhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li, Junjie Huang, Yintong Huo, Pinjia He, Jiazhen Gu, and Michael R. Lyu. 2023. LLMParser: A LLM-based Log Parsing Framework.arXiv preprint arXiv:2310.01796(2023)

    2023 arXiv

  15. [20]

    Zhihan Jiang, Jinyang Liu, Junjie Huang, Yichen Li, Yintong Huo, Jiazhen Gu, Zhuangbin Chen, Jieming Zhu, and Michael R. Lyu. 2023. A Large-scale Bench- mark for Log Parsing.arXiv preprint arXiv:2308.10828(2023)

    2023 arXiv

  16. [21]

    Sahil Lal, Nikhil Sardana, and Anil Sureka. 2016. LogOptPlus: Learning to Opti- mize Logging in catch and if Programming Constructs. In40th IEEE Computer Software and Applications Conference (COMPSAC), Vol. 1. 215–220

    2016

  17. [22]

    Heng Li, Zhilei Mao, and David Lo. 2016. Towards Just-in-Time Suggestions for Log Changes.Empirical Software Engineering21, 2 (2016), 621–656

    2016

  18. [23]

    Heng Li, Weiyi Shang, and Ahmed E. Hassan. 2017. Which Log Level Should Developers Choose for a New Logging Statement?Empirical Software Engineering 22 (2017), 1684–1716

    2017

  19. [25]

    Yichen Li, Yintong Huo, Zhihan Jiang, Renyi Zhong, Pinjia He, Yuxin Su, and Michael R. Lyu. 2023. Exploring the Effectiveness of LLMs in Automated Logging Generation: An Empirical Study.arXiv preprint arXiv:2307.05950(2023)

    2023 arXiv

  20. [26]

    Yichen Li, Yintong Huo, Renyi Zhong, Zhihan Jiang, Jing Liu, Jiang Huang, Jiazhen Gu, Pinjia He, and Michael R. Lyu. 2024. Go Static: Contextualized Logging Statement Generation.Proceedings of the ACM on Software Engineering 1, FSE (2024), 609–630

    2024

  21. [27]

    Zhenhao Li, Tse-Hsun Chen, Jingwei Yang, and Weiyi Shang. 2021. Studying Duplicate Logging Statements and Their Relationships with Code Clones.IEEE Transactions on Software Engineering(2021), 2476–2494

    2021

  22. [28]

    Zhenhao Li, Tse-Hsun Chen, Jing Yang, and Weiyi Shang. 2023. Are They All Good? Studying Practitioners’ Expectations on the Readability of Log Messages. In38th IEEE/ACM International Conference on Automated Software Engineering (ASE). 1–12

    2023

  23. [29]

    Zhenhao Li, Tse-Hsun Peter Chen, and Weiyi Shang. 2020. Where Shall We Log? Studying and Suggesting Logging Locations in Code Blocks. In35th IEEE/ACM International Conference on Automated Software Engineering (ASE). 361–372

    2020

  24. [30]

    Zhenhao Li, Heng Li, Tse-Hsun Chen, and Weiyi Shang. 2021. DeepLV: Suggesting Log Levels Using Ordinal Based Neural Networks. In43rd IEEE/ACM International Conference on Software Engineering (ICSE). 1461–1472

    2021

  25. [31]

    Zhenhao Li and Weiyi Shang. 2014. The Game of Twenty Questions: Do You Know Where to Log?. In11th Working Conference on Mining Software Repositories (MSR). 146–156

    2014

  26. [32]

    Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. InWorkshop on Text Summarization Branches Out. 74–81

    2004

  27. [33]

    Jiahao Liu, Jun Zeng, Xiang Wang, Kaihang Ji, and Zhenkai Liang. 2022. TeLL: Log Level Suggestions via Modeling Multi-Level Code Block Information. In31st ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA). 27–38

    2022

  28. [34]

    Hassan, and Sen Li

    Zihan Liu, Xiaoning Xia, David Lo, Zhenchang Xing, Ahmed E. Hassan, and Sen Li. 2019. Which Variables Should I Log?IEEE Transactions on Software Engineering(2019), 2012–2031

    2019

  29. [35]

    Robert C. Martin. 2002.Agile Software Development: Principles, Patterns, and Practices. Prentice Hall, Upper Saddle River, NJ

    2002

  30. [36]

    Antonio Mastropaolo. 2023. LANCE2.0. https://github.com/antonio-mastropaolo/ automating-logging-acitivities

    2023

  31. [37]

    Antonio Mastropaolo, Luca Pascarella, and Gabriele Bavota. 2022. Using Deep Learning to Generate Complete Log Statements. In44th International Conference on Software Engineering (ICSE). 2279–2290

    2022

  32. [38]

    Eduardo Mendes and Fabio Petrillo. 2021. Log Severity Levels Matter: A Multivo- cal Mapping Study.arXiv preprint arXiv:2109.01192(2021)

    2021 arXiv

  33. [39]

    OpenAI. 2024. o3-mini. https://platform.openai.com/. Accessed July 2025

    2024

  34. [40]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A Method for Automatic Evaluation of Machine Translation. In40th Annual Meeting of the Association for Computational Linguistics (ACL). 311–318

    2002

  35. [41]

    Kunal Patel, José Faccin, Ameél Hamou-Lhadj, and Iury Nunes. 2022. The Sense of Logging in the Linux Kernel.Empirical Software Engineering27, 6 (2022), 153

    2022

  36. [42]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.Journal of Machine Learning Research21, 1 (2020), 5485–5551

    2020

  37. [43]

    Guoping Rong, Shenghui Gu, He Zhang, Dong Shao, and Wanggen Liu. 2018. How Is Logging Practice Implemented in Open Source Software Projects? A Preliminary Exploration. In25th Australasian Software Engineering Conference (ASWEC). 171–180

    2018

  38. [44]

    N Sasirekha, A Edwin Robert, and Dr M Hemalatha. 2011. Program slicing techniques and its applications.arXiv preprint arXiv:1108.1352(2011)

    2011 arXiv

  39. [45]

    Raja Vallée-Rai, Phong Co, Etienne Gagnon, Laurie Hendren, Patrick Lam, and Vijay Sundaresan. 2010. Soot: A Java Bytecode Optimization Framework. In CASCON First Decade High Impact Papers. 214–224

    2010

  40. [46]

    Wegman and F

    Mark N. Wegman and F. Kenneth Zadeck. 1991. Constant Propagation with Conditional Branches.ACM Transactions on Programming Languages and Systems 13, 2 (1991), 181–210

    1991

  41. [47]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of Thought Prompting Elicits Reasoning in Large Language Models.arXiv preprint arXiv:2201.11903(2022)

    2022 arXiv

  42. [50]

    Fermin Yamaguchi, Nicolai Golde, Daniel Arp, and Konrad Rieck. 2014. Model- ing and Discovering Vulnerabilities with Code Property Graphs. In33rd IEEE Symposium on Security and Privacy (S&P). 590–604

    2014

  43. [51]

    Jie Yang, Yuan Dang, and Dongmei Zhang. 2018. Predicting Logging Levels Using Software Development Features. In24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 2074–2083

    2018

  44. [52]

    de Pádua, Weiyi Shang, Steve Sporea, Andrei Toma, and Sarah Sajedi

    Kundi Yao, Guilherme B. de Pádua, Weiyi Shang, Steve Sporea, Andrei Toma, and Sarah Sajedi. 2018. Log4perf: Suggesting logging locations for web-based systems’ performance monitoring. InProceedings of the 2018 ACM/SPEC International Conference on Performance Engineering (ICPE)...

    2018

  45. [53]

    Lee, Xiaoming Tang, Yuanyuan Zhou, and Stefan Savage

    Ding Yuan, Soyeon Park, Peng Huang, Yang Liu, Michael M. Lee, Xiaoming Tang, Yuanyuan Zhou, and Stefan Savage. 2012. Be Conservative: Enhancing Failure Diagnosis with Proactive Logging. In10th USENIX Symposium on Operating Systems Design and Implementation (OSDI). 293–306

    2012

  46. [54]

    Ding Yuan, Soyeon Park, and Yuanyuan Zhou. 2012. Characterizing Logging Practices in Open-Source Software. In34th International Conference on Software Engineering (ICSE). 102–112

    2012

  47. [55]

    Ding Yuan, Jing Zheng, Soyeon Park, Yuanyuan Zhou, and Stefan Savage. 2012. Improving Software Diagnosability via Log Enhancement.ACM Transactions on Computer Systems30, 1 (2012), 4

    2012

  48. [56]

    He Zhang, Shenghui Gu, Guoping Rong, and Wanggen Liu. 2020. Automatically Generating Descriptive Texts in Logging Statements: How Far Are We?. In18th Asian Symposium on Programming Languages and Systems (APLAS) (Lecture Notes in Computer Science, Vol. 12470). Springer, 311–331

    2020

  49. [58]

    Weinberger, and Yoav Artzi

    Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi

  50. [59]

    Yang Zhang, Xiaosong Chang, Lining Fang, and Yifan Lu. 2023. DeepLog: Deep- Learning-Based Log Recommendation. In2023 IEEE/ACM 45th International Con- ference on Software Engineering: Companion Proceedings (ICSE-Companion). 88– 92

    2023

  51. [60]

    Xu Zhao, Kirk Rodrigues, Yu Luo, Michael Stumm, Ding Yuan, and Yuanyuan Zhou. 2017. Log20: Fully automated optimal placement of log printing statements under specified overhead threshold. InProceedings of the 26th Symposium on Operating Systems Principles (SOSP). 565–581

    2017

  52. [61]

    Lyu, and Dongmei Zhang

    Jieming Zhu, Peng He, Qiang Fu, Hongyu Zhang, Michael R. Lyu, and Dongmei Zhang. 2015. Learning to Log: Helping Developers Make Informed Logging Decisions. In37th IEEE/ACM International Conference on Software Engineering (ICSE), Vol. 1. 415–425

    2015

  53. [2020]

    In8th International Conference on Learning Representations (ICLR)

    BERTScore: Evaluating Text Generation with BERT. In8th International Conference on Learning Representations (ICLR)

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.