Pith. sign in

REVIEW 4 major objections 6 minor 75 references

From Noise to Knowledge: Interactive Summaries for Developer Alerts

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CLARITY turns static-analysis warning review into an interactive, rule-learning sensemaking process.

desk verdict Worth reading for the tool and user study; the simulation's headline speedup is largely definitional and the abstract overstates speed. read the letter →

arxiv 2508.07169 v1 pith:HMNKFIZF submitted 2025-08-10 cs.SE

Burak Yetiştiren , Hong Jin Kang , Miryung Kim This is my paper
classification cs.SE
keywords staticanalysiswarningsinteractivesensemakinginductivelogicprogrammingactivelearningwarningtriagesummaryrulescodeexpressionsuserstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Static-analysis tools flood developers with warnings, many of them uninteresting, and current interfaces show them one at a time. CLARITY proposes to make warning triage a sensemaking activity: as a developer marks a few warnings as interesting or uninteresting and optionally highlights a code expression, a rule-inference engine builds summary rules that group similar warnings by package, class, subtype, return type, used fields, invoked methods, and highlighted expressions. The paper's claim is that this rule-level abstraction is learnable from a handful of labels and genuinely changes how developers reason about alerts. In a within-subject study with 14 participants, CLARITY users articulated more common symptoms of uninteresting warnings, wrote longer explanations, and reported higher confidence and lower mental demand than a one-by-one baseline; a simulation on null-pointer warnings from one of the study's Java projects indicates that rule-level feedback reaches 80% rule alignment in 11.8 interactions on average versus 17.8 with instance-level feedback only. The authors read these results as evidence that interactive, user-guided summarization—not just filtering or ranking—is a viable way to make bug-finding tools digestible.

What carries the argument

The mechanism is inductive logic programming (ILP) over a background knowledge base. CLARITY seeds the knowledge base with containment, subtyping, and API-signature facts for each warning's implicated code, and augments it with facts about expressions the user highlights. An off-the-shelf logic-program solver searches for a hypothesis: an assignment of predicates to rules such that every rule matches as many labeled uninteresting warnings as possible while matching no labeled interesting warning, breaking ties toward generality. The rules are presented to the user with per-rule warning distributions, and the user can either label individual warnings or apply a rule to all matched warnings, c

What would settle it

Give CLARITY a corpus of warnings where the uninteresting ones share a common cause invisible to its predicates—for example, a comment marker, a project configuration flag, or a dataflow property such as 'this value comes from a deserializer'—and measure rule alignment under a user who labels by that cause. If alignment cannot be driven above the 80% threshold no matter how many rule-level interactions are allowed, the vocabulary premise fails.

Watch

Extended reading notes

Core claim

The central claim is that warnings from a bug-finding tool can be made sense of by learning a small set of summary rules from the user's own judgments, rather than by presenting each warning in isolation. A rule is a conjunction of logic predicates drawn from a seeded background knowledge of containment (package, class), typing (subtype/interfaces), and API signatures (return type, used fields), plus predicates for code expressions that the user explicitly highlights. The learner is required to cover the warnings the user has marked uninteresting, avoid warnings marked interesting, and maximize the number of still-uninspected warnings matched. The paper reports that this approach yields fast

Load-bearing premise

The load-bearing premise is that the things that make a warning uninteresting can be expressed with the small fixed vocabulary CLARITY starts from—package, class, return type, used fields, subtyping, and any code expression a user happens to highlight. If the real common cause lives in a feature outside that vocabulary, CLARITY's rules cannot group those warnings, and the improvement disappears. The paper acknowledges part of this in its design by seeding only containment/typ

Editorial extensions

If this is right

  • Warning review can shift from per-alert inspection to group-level reasoning: one rule can stand in for many warnings that share a package, subtype, or API call.
  • Rule-level feedback is a real efficiency lever: aligning inferred rules with a simulated user's labels takes 11.8 interactions on average with rule-level feedback versus 17.8 with instance-level feedback only.
  • Because 14 participants produced 55 rules with at least one unique rule each, a one-size-fits-all suppression policy is unlikely to match individual developers' mental models.
  • CLARITY-style rules can serve as codified, shareable suppression policies that capture why a team ignores certain warnings, not just which warnings to ignore.
  • Sensemaking improvements come with a cost: participants took longer and examined fewer warnings, suggesting the benefit is in understanding, not raw throughput.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's null-pointer and resource-leak settings, the same rule-induction loop should apply to any warning class whose false positives are localized by package, type, or API usage; the paper conjectures this but does not test it.
  • The design implicitly bets on the user's ability to notice salient expressions: expression predicates are harvested only from highlighted code, so a user who does not highlight misses those groupings. A tool could test this by prompting users for highlights or mining frequent expressions automatically.
  • Rule alignment in the simulation assumes the simulated user's labels are themselves expressible as conjunctions of the available predicates; in practice some users may group warnings by criteria no predicate can state, which would make perfect alignment unattainable.
  • The user-study result that participants examined fewer warnings under CLARITY suggests that measuring success by warnings-per-minute would miss the point; a better productivity metric may be number of distinct root causes understood per unit time.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents CLARITY, an interactive tool that uses active learning and inductive logic programming to summarize static-analyzer warnings into customizable rules. The evaluation comprises a within-subject user study with 14 participants, comparing CLARITY to a one-by-one warning-list baseline, and a simulation comparing rule-level feedback with instance-level feedback. The authors report that CLARITY yields higher confidence and better articulation of commonalities, lower perceived mental demand, and that rule-level feedback aligns inferred rules in fewer interactions (11.8 vs. 17.8).

Significance. If the simulation result were interpretable, the paper would make a useful contribution to interactive warning triage by combining active learning with ILP over code structure. The user study is a reasonable within-subject design with a replication package and produces statistically significant effects on confidence and mental demand; the observation of high individual variation in rules is a useful empirical finding. The main quantitative support for faster alignment is, however, not currently interpretable, which caps the significance of the paper.

major comments (4)
  1. [V-A, V-B] The simulation's ground-truth labels are never defined. Section V-A says the simulated user 'makes a decision for each of the selected warnings,' but the paper does not say what the correct label is for each warning. The alignment metric in V-B is defined relative to 'the user's labeling,' so the simulation cannot be reproduced and the alignment curves in Figures 6-7 are not interpretable. Please specify the oracle used to label warnings as interesting/uninteresting (e.g., labels from the user study, historical fixes, or a fixed rule) and explain how the simulated user's decisions are generated.
  2. [V-C] The comparison between rule-level and instance-level feedback conflates interaction count with the number of labels provided. In the rule-level condition (p=1), one interaction labels every warning entailed by the rule; in the instance-level condition (p=0), one interaction labels exactly one warning. Since alignment measures the fraction of a rule's matched warnings consistent with user labels, the rule-level condition has a large labeling-volume advantage by construction. The 11.8 vs. 17.8 result is therefore not evidence that the ILP inference is better. To support the claim, control for the number of labels (e.g., compare performance after the same number of labeled warnings) or report per-label alignment efficiency.
  3. [Abstract and Section V-C / VI] The abstract claims users 'articulated root causes ... faster' and that '11.8 interactions are needed to align all inferred rules.' No articulation-time measurement is reported; Section IV-B reports a 13% increase in number of common symptoms and longer written descriptions, and Section VI explicitly states participants took longer and inspected 24% fewer warnings. The simulation in V-C reports reaching the 80% alignment threshold (alpha=0.8) at 11.8 interactions, not 'all inferred rules.' The Introduction also says 'within 18 iterations, rule alignment becomes 100%.' These statements are inconsistent and should be reconciled; 'faster' should be removed unless time-to-articulation is measured.
  4. [III-B] The background knowledge is initialized only with containment, type, and API-signature predicates; code expressions are added only when the user highlights them. The paper's claim that CLARITY 'surfaces common symptoms' is therefore limited by this vocabulary. If the actual reason a warning is uninteresting is not captured by package/class/return type/fields/subtyping or a highlighted expression, the rules cannot group those warnings. The user study covers only null-pointer dereference warnings, which may be unusually well matched to these features; the generalization statement in Section VI is not directly supported. This should be presented as a limitation and, if possible, tested with other warning types or an expanded vocabulary.
minor comments (6)
  1. [IV-B / Introduction] Section IV-B reports '13% more common symptoms' (1.53 vs. 1.35), while the Introduction says '10% more'; please harmonize the numbers.
  2. [V-A] The probability variable p is rendered as '�' in several places (e.g., '� � �', '� � � ��'). Define p with a readable symbol and ensure figures use consistent notation.
  3. [Figures 6-7] Axis labels and some caption text are garbled in the typeset version; provide high-resolution figures with legible labels.
  4. [References] Reference [66] appears to duplicate [28] (same title) but with a different year ('2014' vs. '2024'); verify the citation.
  5. [VI] Section VI states participants 'took longer' but no time measurements are reported; report average task completion times or qualify the statement.
  6. [IV-Baseline] The baseline is described as a 'downgraded version of CLARITY' rather than an unmodified existing tool. State more explicitly how the one-by-one baseline differs from CLARITY's interface and whether this could bias the comparison.

Circularity Check

1 steps flagged · score 6.0 of 10

Simulation's faster-alignment result reduces by construction: rule-level feedback labels all matched warnings at once, so the metric makes rule-level alignment definitionally faster.

  1. self definitional [Section V-A (Simulated Behavior), Section V-B (Metric), Section V-C (Results)]
    "With a probability ρ, a user provides feedback at the rule level. For example, ρ = 0.3 means that a user provides rule-level feedback 30% of interactions, i.e., marking all warnings entailed by the rule as uninteresting. For providing feedback, we simulate a user making a decision for each of the selected warnings. If a rule r matches 5 warnings and 4 of them have the same label assignment as the user's labeling, we say r is 80% aligned with the user's labeling."

    Rule-level feedback is defined as labeling every warning entailed by a rule in a single interaction, and Alignment is defined as agreement with the user's labeling. Therefore, once a rule receives rule-level feedback, all warnings it matches have the same label as the user's labeling, making that rule 100% aligned by construction. In contrast, instance-level feedback changes only one warning per interaction, so a rule can approach 100% alignment only one warning at a time. The reported result—80% alignment at 11.8 interactions for ρ=1 vs 17.8 for ρ=0—therefore measures the number of labels injected per interaction, not the quality of the ILP inference. The speedup is forced by the definitions of the feedback mode and the metric.

full rationale

The central product claim—that CLARITY improves sensemaking—rests on two evaluations. The user study (Section IV) is an empirical within-subject comparison whose outcomes (articulation, confidence, TLX) are independent of the algorithm's definitions; that part is not circular. The simulation (Section V), however, is circular in its core comparison: rule-level feedback is defined as marking all warnings entailed by a rule in a single interaction, and Alignment is defined as agreement with the user's labeling. Hence any rule receiving rule-level feedback is 100% aligned with the simulated user's labels immediately, by construction, while instance-level feedback can only change one warning per interaction. Comparing first-crossing times on an interaction count therefore measures label throughput, not rule-learning quality; the 11.8 vs 17.8 result is a definitional artifact. The simulation also lacks an externally specified ground truth for the simulated user's decisions, so 'alignment with user feedback' is self-referential. Additionally, the abstract's 'align all inferred rules' at 11.8 interactions conflicts with V-C's 80% threshold. These are correctness/validity problems, not additional circularity. No load-bearing self-citation or ansatz-smuggling was found; the ILP engine is off-the-shelf and the self-citations are to prior active-learning work, which is not used to force the present result. Score 6: one central quantitative 'prediction' reduces by construction.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The approach rests on the assumption that structural predicates and user-highlighted code expressions capture the features that determine whether warnings are interesting. The simulation additionally assumes the simulated user's labels are a correct ground truth, which is not described. No new physical or conceptual entities are introduced beyond the CLARITY tool itself.

free parameters (2)
  • alignment threshold tau = 80%
    Chosen by the authors to define an 'aligned' rule; the reported interaction counts (11.8 vs 17.8) depend on this threshold.
  • rule-level feedback probability p = 0, 0.3, 1
    Simulation conditions; the headline 11.8 vs 17.8 comparison uses p=1 vs p=0.
assumptions (3)
  • domain assumption Warnings that share structural predicates (package, class, return type, fields, subtyping) or user-highlighted code expressions are likely to share the same interestingness label.
    Central to CLARITY's rule design (Section III-C). If false, rules cannot group warnings meaningfully.
  • domain assumption The ground-truth labels used to simulate the user's feedback in Section V are correct and representative.
    The paper does not state how these labels were obtained; the alignment metric and all reported interaction counts depend on them.
  • standard math The inductive logic programming solver is sound and the hypothesis space is appropriately restricted.
    Algorithm 1 relies on an off-the-shelf logic program solver (gringo/clasp), with no formal verification or detailed hypothesis-space specification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Noise to Knowledge: Interactive Summaries for Developer Alerts." pith.science (2026). https://pith.science/paper/HMNKFIZF

@misc{pith2026250807169,
  author       = {Pith},
  title        = {Pith review of: From Noise to Knowledge: Interactive Summaries for Developer Alerts},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HMNKFIZF}},
  note         = {Machine review of arXiv:2508.07169}
}
read the original abstract

Programmers using bug-finding tools often review their reported warnings one by one. Based on the insight that identifying recurring themes and relationships can enhance the cognitive process of sensemaking, we propose CLARITY, which supports interpreting tool-generated warnings through interactive inquiry. CLARITY derives summary rules for custom grouping of related warnings with active feedback. As users mark warnings as interesting or uninteresting, CLARITY's rule inference algorithm surfaces common symptoms, highlighting structural similarities in containment, subtyping, invoked methods, accessed fields, and expressions. We demonstrate CLARITY on Infer and SpotBugs warnings across two mature Java projects. In a within-subject user study with 14 participants, users articulated root causes for similar uninteresting warnings faster and with more confidence using CLARITY. We observed significant individual variation in desired grouping, reinforcing the need for customizable sensemaking. Simulation shows that with rule-level feedback, only 11.8 interactions are needed on average to align all inferred rules with a simulated user's labels (vs. 17.8 without). Our evaluation suggests that CLARITY's active learning-based summarization enhances interactive warning sensemaking.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

75 extracted references · 62 canonical work pages

  1. [1]

    Using static analysis to find bugs,

    N. Ayewah, W. Pugh, D. Hovemeyer, J. D. Morgenthaler, and J. Penix, “Using static analysis to find bugs,” IEEE software, vol. 25, no. 5, pp. 22–29, 2008

    work page 2008

  2. [2]

    Moving fast with software verification,

    C. Calcagno, D. Distefano, J. Dubreil, D. Gabi, P. Hooimeijer, M. Luca, P. O’Hearn, I. Papakonstantinou, J. Purbrick, and D. Rodriguez, “Moving fast with software verification,” in NASA Formal Methods Symposium. Springer, 2015, pp. 3–11

    work page 2015

  3. [3]

    Why don’t software developers use static analysis tools to find bugs?

    B. Johnson, Y . Song, E. Murphy-Hill, and R. Bowdidge, “Why don’t software developers use static analysis tools to find bugs?” in 2013 35th International Conference on Software Engineering (ICSE). IEEE, 2013, pp. 672–681

    2013

  4. [4]

    What developers want and need from program analysis: an empirical study,

    M. Christakis and C. Bird, “What developers want and need from program analysis: an empirical study,” in Proceedings of the 31st IEEE/ACM international conference on automated software engineering, 2016, pp. 332–343

    work page 2016

  5. [5]

    Efindbugs: Effective error ranking for findbugs,

    H. Shen, J. Fang, and J. Zhao, “Efindbugs: Effective error ranking for findbugs,” in Verification and Validation 2011 Fourth IEEE International Conference on Software Testing, Mar. 2011, p. 299–308. [Online]. Available: https://ieeexplore.ieee.org/document/5770619

    work page arXiv 2011

  6. [6]

    Designing uis for static analysis tools: Evaluating tool design guidelines with swan,

    D. Tiganov, L. Nguyen Quang Do, and K. Ali, “Designing uis for static analysis tools: Evaluating tool design guidelines with swan,” Queue, vol. 19, no. 4, pp. Pages 40:97–Pages 40:118, Sep. 2021. [Online]. Available: https://dl.acm.org/doi/10.1145/3487019.3487026

    work page arXiv 2021

  7. [7]

    How open source projects use static code analysis tools in continuous integration pipelines,

    F. Zampetti, S. Scalabrino, R. Oliveto, G. Canfora, and M. Di Penta, “How open source projects use static code analysis tools in continuous integration pipelines,” in 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR). IEEE, 2017, pp. 334–344

    work page 2017

  8. [8]

    Mar. 2024. [Online]. Available: https://github.com/alibaba/nacos

    work page 2024

Show all 75 references
  1. [9]

    Why can’t johnny fix vulnerabilities: A usability evaluation of static analysis tools for security,

    J. Smith, L. N. Q. Do, and E. Murphy-Hill, “Why can’t johnny fix vulnerabilities: A usability evaluation of static analysis tools for security,” in Sixteenth Symposium on Usable Privacy and Security (SOUPS 2020). USENIX Association, Aug. 2020, pp. 221–

    2020

  2. [10]

    Mitigating false positive static analysis warn- ings: Progress, challenges, and opportunities,

    Z. Guo, T. Tan, S. Liu, X. Liu, W. Lai, Y . Yang, Y . Li, L. Chen, W. Dong, and Y . Zhou, “Mitigating false positive static analysis warn- ings: Progress, challenges, and opportunities,” IEEE Transactions on Software Engineering, vol. 49, no. 12, pp. 5154–5188, 2023

    2023

  3. [11]

    Marton, Necessary conditions of learning

    F. Marton, Necessary conditions of learning. Routledge, 2014

    2014

  4. [12]

    Correlation exploita- tion in error ranking,

    T. Kremenek, K. Ashcraft, J. Yang, and D. Engler, “Correlation exploita- tion in error ranking,” ACM SIGSOFT Software Engineering Notes, vol. 29, no. 6, pp. 83–93, 2004

    2004

  5. [13]

    Oct. 2020. [Online]. Available: https://gitlab.com/gitlab-org/gitlab/-/ issues/270461

    2020

  6. [14]

    System properties (java tutorials, essential java classes),

    “System properties (java tutorials, essential java classes),” https://docs. oracle.com/javase/tutorial/essential/environment/sysprop.html, 2018

    2018

  7. [15]

    Deckard: Scalable and accurate tree-based detection of code clones,

    L. Jiang, G. Misherghi, Z. Su, and S. Glondu, “Deckard: Scalable and accurate tree-based detection of code clones,” in 29th International Conference on Software Engineering (ICSE’07). IEEE, 2007, pp. 96– 105

    2007

  8. [16]

    An empirical study of code clone genealogies,

    M. Kim, V . Sazawal, D. Notkin, and G. Murphy, “An empirical study of code clone genealogies,” in Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, 2005, pp. 187–196

    2005

  9. [17]

    Oreo: Detection of clones in the twilight zone,

    V . Saini, F. Farmahinifarahani, Y . Lu, P. Baldi, and C. V . Lopes, “Oreo: Detection of clones in the twilight zone,” in Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, 2018, ...

    2018

  10. [18]

    Tailoring programs for static analysis via program transformation,

    R. van Tonder and C. L. Goues, “Tailoring programs for static analysis via program transformation,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 824–834

    2020

  11. [19]

    Infer explore,

    “Infer explore,” https://fbinfer.com/man/next/infer-explore.1.html/, 2023

    2023

  12. [20]

    Analyzing temporal api usage patterns,

    G. Uddin, B. Dagenais, and M. P. Robillard, “Analyzing temporal api usage patterns,” in 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011). IEEE, 2011, pp. 456– 459

    2011

  13. [21]

    Rogueone: Detecting rogue updates via differential data-flow analysis using trust domains,

    R. Sofaer, Y . David, M. Kang, J. Yu, Y . Cao, J. Yang, and J. Nieh, “Rogueone: Detecting rogue updates via differential data-flow analysis using trust domains,” in 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE). IEEE Computer Society, 2024, pp. 885–885

    2024

  14. [22]

    Occam’s razor,

    A. Blumer, A. Ehrenfeucht, D. Haussler, and M. K. Warmuth, “Occam’s razor,” Information processing letters, vol. 24, no. 6, pp. 377–380, 1987

    1987

  15. [23]

    Inductive logic programming at 30,

    A. Cropper, S. Duman ˇci´c, R. Evans, and S. H. Muggleton, “Inductive logic programming at 30,” Machine Learning, vol. 111, no. 1, pp. 147– 172, 2022

    2022

  16. [24]

    gringo, clasp, clingo, and iclingo,

    M. Gebser, R. Kaminski, B. Kaufmann, M. Ostrowski, T. Schaub, and S. Thiele, “gringo, clasp, clingo, and iclingo,” 2010

    2010

  17. [25]

    Crossover designs in software engineering experiments: Benefits and perils,

    S. Vegas, C. Apa, and N. Juristo, “Crossover designs in software engineering experiments: Benefits and perils,” IEEE Transactions on Software Engineering, vol. 42, no. 2, p. 120–135, Feb. 2016

    2016

  18. [26]

    Pl and hci: better together,

    S. E. Chasins, E. L. Glassman, and J. Sunshine, “Pl and hci: better together,” Communications of the ACM, vol. 64, no. 8, pp. 98–106, 2021

    2021

  19. [27]

    How to support ml end-user programmers through a conversational agent,

    E. J. Arteaga Garcia, J. F. Nicolaci Pimentel, Z. Feng, M. Gerosa, I. Steinmacher, and A. Sarma, “How to support ml end-user programmers through a conversational agent,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New Yo...

    2024

  20. [28]

    Scaling code pattern inference with interactive what-if analysis,

    H. J. Kang, K. Wang, and M. Kim, “Scaling code pattern inference with interactive what-if analysis,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ser. ICSE ’24. New York, NY , USA: Association for Computing Machinery, Apr. 2024, p. 1–12...

    2024

  21. [29]

    Code coverage criteria for asynchronous programs,

    M. Ganji, S. Alimadadi, and F. Tip, “Code coverage criteria for asynchronous programs,” in Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ser. ESEC/FSE 2023. New York, NY , USA: Association f...

    2023 doi

  22. [30]

    Using annotations for sensemaking about code,

    A. Horvath, B. Myers, A. Macvean, and I. Rahman, “Using annotations for sensemaking about code,” in UIST’22. Bend OR USA: ACM, Oct. 2022, p. 1–16. [Online]. Available: https: //dl.acm.org/doi/10.1145/3526113.3545667

    2022

  23. [32]

    Designchecker: Visual design support for blind and low vision web developers,

    M. Huh and A. Pavel, “Designchecker: Visual design support for blind and low vision web developers,” in Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, ser. UIST ’24. New York, NY , USA: Association for Computing Machinery, Oct. 2024, p....

    2024

  24. [33]

    Selenite: Scaffolding online sensemaking with comprehensive overviews elicited from large language models,

    M. X. Liu, T. Wu, T. Chen, F. M. Li, A. Kittur, and B. A. Myers, “Selenite: Scaffolding online sensemaking with comprehensive overviews elicited from large language models,” in Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, ser. CHI ’24. New York...

    2024

  25. [34]

    Synergi: A mixed-initiative system for scholarly synthesis and sensemaking,

    H. B. Kang, T. Wu, J. C. Chang, and A. Kittur, “Synergi: A mixed-initiative system for scholarly synthesis and sensemaking,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, ser. UIST ’23. New York, NY , USA: Association for Computing ...

    2023

  26. [35]

    Beyond code generation: Llm-supported exploration of the program design space,

    J. D. Zamfirescu-Pereira, E. Jun, M. Terry, Q. Yang, and B. Hartmann, “Beyond code generation: Llm-supported exploration of the program design space,” Mar. 2025, arXiv:2503.06911 [cs]. [Online]. Available: http://arxiv.org/abs/2503.06911

    2025 arXiv

  27. [36]

    Prompting for discovery: Flexible sense-making for ai art-making with dreamsheets,

    S. G. Almeda, J. Zamfirescu-Pereira, K. W. Kim, P. Mani Rathnam, and B. Hartmann, “Prompting for discovery: Flexible sense-making for ai art-making with dreamsheets,” in Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, ser. CHI ’24. New York, NY , ...

    2024

  28. [37]

    Perfection not required? human-ai partnerships in code translation,

    J. D. Weisz, M. Muller, S. Houde, J. Richards, S. I. Ross, F. Martinez, M. Agarwal, and K. Talamadupula, “Perfection not required? human-ai partnerships in code translation,” in 26th International Conference on Intelligent User Interfaces, Apr. 2021, p. 402–412, arXiv:2104.038...

    2021 arXiv

  29. [38]

    Mar. 2024. [Online]. Available: https://github.com/apache/lucene-solr

    2024

  30. [39]

    Learning to reduce false positives in analytic bug detectors,

    A. Kharkar, R. Z. Moghaddam, M. Jin, X. Liu, X. Shi, C. Clement, and N. Sundaresan, “Learning to reduce false positives in analytic bug detectors,” in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 1307–1316

    2022

  31. [40]

    Is there a

    J. Wang, S. Wang, and Q. Wang, “Is there a” golden” feature set for static warning identification? an experimental evaluation,” in Proceedings of the 12th ACM/IEEE international symposium on empirical software engineering and measurement, 2018, pp. 1–10

    2018

  32. [41]

    Understanding static code warnings: An incremental ai approach,

    X. Yang, Z. Yu, J. Wang, and T. Menzies, “Understanding static code warnings: An incremental ai approach,” Expert Systems with Applications, vol. 167, p. 114134, 2021

    2021

  33. [42]

    Active learning of discriminative subgraph patterns for api misuse detection,

    H. J. Kang and D. Lo, “Active learning of discriminative subgraph patterns for api misuse detection,” IEEE Transactions on Software Engineering, vol. 48, no. 8, pp. 2761–2783, 2021

    2021

  34. [43]

    How to find actionable static analysis warnings: A case study with findbugs,

    R. Yedida, H. J. Kang, H. Tu, X. Yang, D. Lo, and T. Menzies, “How to find actionable static analysis warnings: A case study with findbugs,” IEEE Transactions on Software Engineering, 2023

    2023

  35. [44]

    Available: https://github.com/claritytool/CLARITY/blob/main/ code/data/user study/All%20Rules.pdf

    [Online]. Available: https://github.com/claritytool/CLARITY/blob/main/ code/data/user study/All%20Rules.pdf

  36. [45]

    Should computers be easy to use? questioning the doctrine of simplicity in user interface design,

    A. Sarkar, “Should computers be easy to use? questioning the doctrine of simplicity in user interface design,” in Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–10

    2023

  37. [46]

    Crossover designs in software engineering experiments: Benefits and perils,

    S. Vegas, C. Apa, and N. Juristo, “Crossover designs in software engineering experiments: Benefits and perils,” IEEE Transactions on Software Engineering, vol. 42, no. 2, pp. 120–135, 2015

    2015

  38. [47]

    Mesh: Scaffolding comparison tables for online decision making,

    J. C. Chang, N. Hahn, and A. Kittur, “Mesh: Scaffolding comparison tables for online decision making,” in Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, ser. UIST ’20. New York, NY , USA: Association for Computing Machinery, Oct. 2020, ...

    2020

  39. [48]

    Unakite: Scaffolding developers’ decision-making using the web,

    M. X. Liu, J. Hsieh, N. Hahn, A. Zhou, E. Deng, S. Burley, C. Taylor, A. Kittur, and B. A. Myers, “Unakite: Scaffolding developers’ decision-making using the web,” in Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology, ser. UIST ’19. New Yor...

    2019

  40. [49]

    The cost structure of sensemaking,

    D. M. Russell, M. J. Stefik, P. Pirolli, and S. K. Card, “The cost structure of sensemaking,” in Proceedings of the INTERACT ’93 and CHI ’93 Conference on Human Factors in Computing Systems, ser. CHI ’93. New York, NY , USA: Association for Computing Machinery, May 1993, p. 26...

    1993

  41. [50]

    Generation probabilities are not enough: Uncertainty highlighting in ai code completions,

    H. Vasconcelos, G. Bansal, A. Fourney, Q. V . Liao, and J. W. Vaughan, “Generation probabilities are not enough: Uncertainty highlighting in ai code completions,” ACM Transactions on Computer-Human Interaction, p. 3702320, Oct. 2024, arXiv:2302.07248 [cs]

    2024 arXiv

  42. [51]

    Calibration and correctness of language models for code,

    C. Spiess, D. Gros, K. S. Pai, M. Pradel, M. R. I. Rabin, A. Alipour, S. Jha, P. Devanbu, and T. Ahmed, “Calibration and correctness of language models for code,” no. arXiv:2402.02047, Aug. 2024, arXiv:2402.02047 [cs]. [Online]. Available: http://arxiv.org/abs/2402. 02047

    2024 arXiv

  43. [52]

    Prioritizing warning categories by analyzing software history,

    S. Kim and M. D. Ernst, “Prioritizing warning categories by analyzing software history,” in Fourth International Workshop on Mining Software Repositories (MSR’07: ICSE Workshops 2007). IEEE, 2007, pp. 27– 27

    2007

  44. [53]

    Finding patterns in static analysis alerts: improving actionable alert ranking,

    Q. Hanam, L. Tan, R. Holmes, and P. Lam, “Finding patterns in static analysis alerts: improving actionable alert ranking,” in Proceedings of the 11th working conference on mining software repositories, 2014, pp. 152–161

    2014

  45. [54]

    A model building process for identifying actionable static analysis alerts,

    S. Heckman and L. Williams, “A model building process for identifying actionable static analysis alerts,” in 2009 International conference on software testing verification and validation. IEEE, 2009, pp. 161–170

    2009

  46. [55]

    Predicting accurate and actionable static analysis warnings: an experimental approach,

    J. R. Ruthruff, J. Penix, J. D. Morgenthaler, S. Elbaum, and G. Rother- mel, “Predicting accurate and actionable static analysis warnings: an experimental approach,” in Proceedings of the 30th international conference on Software engineering, 2008, pp. 341–350

    2008

  47. [56]

    Efindbugs: Effective error ranking for findbugs,

    H. Shen, J. Fang, and J. Zhao, “Efindbugs: Effective error ranking for findbugs,” in 2011 Fourth IEEE International conference on software testing, verification and validation. IEEE, 2011, pp. 299–308

    2011

  48. [57]

    Detecting false alarms from automatic static analysis tools: How far are we?

    H. J. Kang, K. L. Aw, and D. Lo, “Detecting false alarms from automatic static analysis tools: How far are we?” in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 698–709

    2022

  49. [58]

    From quick fixes to slow fixes: Reimagining static analysis resolutions to enable design space exploration,

    T. Barik, Y . Song, B. Johnson, and E. Murphy-Hill, “From quick fixes to slow fixes: Reimagining static analysis resolutions to enable design space exploration,” in 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE, 2016, pp. 211–221

    2016

  50. [59]

    A user-guided approach to program analysis,

    R. Mangal, X. Zhang, A. V . Nori, and M. Naik, “A user-guided approach to program analysis,” in Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, 2015, pp. 462–473

    2015

  51. [60]

    Uav: Warnings from multiple automated static analysis tools at a glance,

    T. Buckers, C. Cao, M. Doesburg, B. Gong, S. Wang, M. Beller, and A. Zaidman, “Uav: Warnings from multiple automated static analysis tools at a glance,” in 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER). IEEE, 2017, pp. 472–476

    2017

  52. [61]

    Reorganizing and optimizing post- inspection on suspicious bug reports in path-sensitive analysis,

    X. Ma, J. Yan, J. Yan, and J. Zhang, “Reorganizing and optimizing post- inspection on suspicious bug reports in path-sensitive analysis,” in 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS). IEEE, 2019, pp. 260–271

    2019

  53. [62]

    Cause points analysis for effective handling of alarms,

    T. Muske and U. P. Khedker, “Cause points analysis for effective handling of alarms,” in 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE). IEEE, 2016, pp. 173–184

    2016

  54. [63]

    Interactive code review for systematic changes,

    T. Zhang, M. Song, J. Pinedo, and M. Kim, “Interactive code review for systematic changes,” in 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 1. IEEE, 2015, pp. 111– 122

    2015

  55. [64]

    Active inductive logic programming for code search,

    A. Sivaraman, T. Zhang, G. Van den Broeck, and M. Kim, “Active inductive logic programming for code search,” in 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE). IEEE, 2019, pp. 292–303

    2019

  56. [65]

    Synthesizing conjunctive queries for code search,

    C. Wang, P. Yao, W. Tang, G. Fan, and C. Zhang, “Synthesizing conjunctive queries for code search,” in 37th European Conference on Object-Oriented Programming, 2023

    2023

  57. [66]

    Scaling code pattern inference with interactive what-if analysis,

    H. J. Kang, K. Wang, and M. Kim, “Scaling code pattern inference with interactive what-if analysis,” in IEEE/ACM International Conference on Software Engineering (ICSE), 2014

    2014

  58. [67]

    Synthesizing code quality rules from examples,

    P. Garg and S. H. Sengamedu, “Synthesizing code quality rules from examples,” Proceedings of the ACM on Programming Languages, vol. 6, no. OOPSLA2, pp. 1757–1787, 2022

    2022

  59. [68]

    Learning quick fixes from code repositories,

    R. Sousa, G. Soares, R. Gheyi, T. Barik, and L. D’Antoni, “Learning quick fixes from code repositories,” in Proceedings of the XXXV Brazilian Symposium on Software Engineering, 2021, pp. 74–83

    2021

  60. [69]

    �SPINFER�: Inferring semantic patches for the linux kernel,

    L. Serrano, V .-A. Nguyen, F. Thung, L. Jiang, D. Lo, J. Lawall, and G. Muller, “ �SPINFER�: Inferring semantic patches for the linux kernel,” in 2020 USENIX Annual Technical Conference (USENIX ATC 20), 2020, pp. 235–248

    2020

  61. [70]

    Tempura: Query analysis with structural templates,

    T. Wu, K. Wongsuphasawat, D. Ren, K. Patel, and C. DuBois, “Tempura: Query analysis with structural templates,” in Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–12

    2020

  62. [71]

    Patat: Human-ai collaborative qualitative coding with explainable interactive rule synthesis,

    S. A. Gebreegziabher, Z. Zhang, X. Tang, Y . Meng, E. L. Glassman, and T. J.-J. Li, “Patat: Human-ai collaborative qualitative coding with explainable interactive rule synthesis,” in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, 2023, pp. 1–19

    2023

  63. [72]

    Supporting serendipity: Opportunities and challenges for human-ai collaboration in qualitative analysis,

    J. A. Jiang, K. Wade, C. Fiesler, and J. R. Brubaker, “Supporting serendipity: Opportunities and challenges for human-ai collaboration in qualitative analysis,” Proceedings of the ACM on Human-Computer Interaction, vol. 5, no. CSCW1, pp. 1–23, 2021

    2021

  64. [73]

    Overcode: Visualizing variation in student solutions to programming problems at scale,

    E. L. Glassman, J. Scott, R. Singh, P. J. Guo, and R. C. Miller, “Overcode: Visualizing variation in student solutions to programming problems at scale,” ACM Transactions on Computer-Human Interaction (TOCHI), vol. 22, no. 2, pp. 1–35, 2015

    2015

  65. [74]

    Visualizing api usage examples at scale,

    E. L. Glassman, T. Zhang, B. Hartmann, and M. Kim, “Visualizing api usage examples at scale,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018, pp. 1–12

    2018

  66. [75]

    Visualizing examples of deep neural networks at scale,

    L. Yan, E. L. Glassman, and T. Zhang, “Visualizing examples of deep neural networks at scale,” in Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021, pp. 1–14

    2021

  67. [238]

    Available: https://www.usenix.org/conference/soups2020/ presentation/smith

    [Online]. Available: https://www.usenix.org/conference/soups2020/ presentation/smith

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.