Pith. sign in

REVIEW 4 major objections 5 minor 59 references

BugRepro: Enhancing Android Bug Reproduction with Domain-Specific Knowledge Integration

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BugRepro claims that adding retrieval-augmented examples and app UI transition graphs lets LLMs extract and replay steps to reproduce Android bugs, succeeding on 96 of 151 real reports.

desk verdict Solid RAG+UTG idea for Android bug reproduction, but the evaluation is compromised by likely train/test overlap in the retrieval corpus, so the headline gains are not credible as presented. read the letter →

arxiv 2505.14528 v2 pith:OGJXSAAS submitted 2025-05-20 cs.SE

Hongrong Yin , Jinhong Huang , Yao Li , Yunwei Dong , Tao Zhang This is my paper · ORCID
classification cs.SE
keywords LargeLanguageModelRetrieval-AugmentedGenerationAndroidAutomationCrashReproductionbugUItransitiongraphS2Rentityextractionstepstoreproduce
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

BugRepro argues that generic LLM ability is not enough for Android bug reproduction; the missing piece is domain-specific knowledge. It adds two knowledge sources: similar bug reports with labeled steps-to-reproduce (S2R) entities retrieved by retrieval-augmented generation, and app-specific UI transition graphs that tell the LLM what each screen and widget does. On 151 real-world bug reports, the paper reports that this combination reproduces 96 crashes, versus 55 for AdbGPT and 38 for ReCDroid, with S2R extraction accuracy 7.57 to 28.89 percentage points higher than the baselines. The claim matters because bug reports are often vague and apps span multiple screens, so an automated assistant that reliably turns text into correct UI actions would shorten debugging. The load-bearing premise is that the retrieval index does not already contain the reports being evaluated.

What carries the argument

The load-bearing objects are the RAG document and the UI transition graph (UTG). The RAG document is an embedding-indexed collection of bug-report sentences annotated with four S2R entity types—action, target element, input value, and scroll direction—retrieved by cosine similarity so the LLM receives few-shot examples at extraction time. The UTG is a directed graph whose nodes are UI states and edges are actions, produced by UI Automator; BugRepro queries the LLM to turn it into a synthesized functionality table of <synthesized functionality, UI states, UI elements> entries that tell the LLM what each widget does and how to reach it. Together these sources supply the domain vocabulary the LLM lacks, while LangChain enforces JSON-formatted action output for execution.

What would settle it

Run the retrieval component against the 151 evaluation reports and count, for each sentence, whether the top retrieved example comes from the same app, the same issue, or a near-duplicate sentence; if a substantial share do, rebuild the RAG index after removing every evaluation report and re-measure extraction accuracy and replay success, and check whether the reported 7.57 to 28.89 percentage point advantage shrinks or disappears.

Watch

Extended reading notes

Core claim

The paper's central claim is that an LLM can reproduce Android crashes from natural-language bug reports if it is given two forms of domain-specific knowledge during prompting. First, a RAG step retrieves, for each sentence in the current report, the most semantically similar previously labeled sentence and its S2R entities, so the prompt contains concrete examples of how to turn phrasing into [action] [component] [value] primitives. Second, when the LLM gets stuck during replay, BugRepro explores the app with UI Automator, builds a UI transition graph, and has the LLM summarize each screen's function and each widget's purpose into a synthesized functionality table that is added to the next prompt. The paper reports that this two-knowledge design is what makes replay succeed across multi-page workflows, and that removing either component lowers the number of successful reproductions from 96 to 80 without RAG or 76 without UI exploration.

Load-bearing premise

The load-bearing premise is that the RAG database, built from the same bug-report collections as the 151-report evaluation set, does not let the LLM see ground-truth S2R entities for the reports it is being tested on.

Editorial extensions

If this is right

  • If the reported numbers hold, a practical workflow exists for automating crash reproduction without per-app training, with 96 of 151 real-world bugs reproduced.
  • The ablation result implies that retrieval examples and UI exploration are independently necessary, not interchangeable, because removing either one reduces successful reproductions by 16 or 20 cases.
  • The extraction accuracy advantage of 7.57 to 11.69 percentage points over AdbGPT suggests that dynamically retrieved examples can outperform a small static hand-built few-shot set.
  • The reported similarity in average reproduction time to AdbGPT (124.7 seconds versus 125.6 seconds) and a 4.3-times speedup over ReCDroid indicates that the added knowledge does not come at a large efficiency cost.
  • The similar S2R accuracy across DeepSeek and GPT-4 suggests the method's gains come from knowledge integration rather than from one specific base model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: the reported extraction gains could be inflated because the RAG database is built from the same bug-report collections (ReCDroid, ScopeDroid, and Su et al.) that later supply the 151-report evaluation set; if a test sentence or close paraphrase is retrievable, the LLM sees its labeled entities as in-context examples, turning generation into lookup.
  • Inference: a clean testable extension is to split the data by repository or report ID, rebuild the RAG index without any evaluation reports, and rerun the extraction and replay experiments under that restriction.
  • Inference: the same two-knowledge recipe—retrieval of annotated examples plus graph-guided exploration—could transfer to other natural-language-to-UI tasks, such as accessibility testing, web automation, or reproducing bugs from video or voice descriptions.
  • Inference: if the leakage concern is resolved, the more durable contribution is the use of LLM-summarized UI transition graphs to resolve multi-screen navigation, which could be tested in isolation by fixing extraction with oracle labels and varying only the exploration knowledge.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes BugRepro, a system for automated Android bug reproduction that combines retrieval-augmented generation (RAG) for extracting steps-to-reproduce (S2R) entities from bug reports with UI-transition-graph-guided exploration for replaying those entities on the app. The RAG component retrieves similar labeled bug-report sentences from a vector database and includes them as in-context examples for an LLM; the replay component builds a synthesized functionality table from UTGs to help the LLM choose UI actions. The paper reports experiments on 151 real-world bug reports, claiming S2R extraction accuracy gains of 7.57 to 28.89 percentage points over ReCDroid and AdbGPT, a bug reproduction success rate of 96/151 versus 55/151 and 38/151, and comparable or better average reproduction time. The central claim is that integrating bug-report-specific and app-specific knowledge into LLM prompts is what drives these gains.

Significance. If the empirical results were trustworthy, BugRepro would be a meaningful step forward in automated bug reproduction, with a practical pipeline that combines RAG and GUI-structured knowledge. The paper has some genuine strengths: it provides a clear end-to-end architecture, an informative ablation study separating the RAG and UI-exploration contributions, and an anonymous GitHub repository with code and datasets, which supports reproducibility. The ablation in Table 9 is a useful check that both components contribute. However, the headline numbers are compromised by a likely overlap between the RAG retrieval corpus and the evaluation set, and the paper provides no statistical significance tests for any comparison. The approach itself is plausible and worth developing, but the current empirical validation does not support the strength of the claims made in the abstract.

major comments (4)
  1. [§3.1.1, §4.1] The RAG retrieval corpus and the evaluation set are not shown to be disjoint. Section 3.1.1 states that the RAG database is built by crawling bug reports from Zhao et al. 2019, Su et al. 2021, Cooper et al. 2021a, and Huang et al. 2023, while Section 4.1 states that the 151-report evaluation set is sourced from the ReCDroid dataset (Zhao et al. 2019), the ScopeDroid dataset (Huang et al. 2023), the AndroR2 dataset (Wendland et al. 2021), and Su et al. 2021. Three of the four sources overlap, and the paper nowhere states that evaluation sentences or near-duplicates were removed from the index. Because the prompt in Section 3.1.2 includes the retrieved sentence together with its human-annotated S2R entities, an exact or near-identical match would supply the ground-truth label as an in-context example, turning what is presented as generation into lookup. The authors must report overlap statistics and re-run all RAG experiments with an index that excludes evaluation data before the gains in Tables 7 and 8 can be interpreted.
  2. [§5.1–§5.4] No statistical significance testing is provided for any comparison. Tables 7, 8, and 10 report point estimates only, with no confidence intervals or paired tests, despite the fact that both extraction and reproduction are naturally paired across the same 151 bug reports. The phrase "significantly outperforms" in the abstract is therefore unsupported. Please add a paired test appropriate to the metrics, such as McNemar's test for per-report reproduction success and a bootstrap or Wilcoxon test for extraction accuracy, and report effect sizes.
  3. [§4.2, §5.2] The time-budget conditions for baselines are not specified. Section 4.2 states that BugRepro uses a 5-minute time budget, but no budget is given for ReCDroid or AdbGPT. Since Table 8 reports ReCDroid's average time as 534.9 seconds, which exceeds 300 seconds, the efficiency comparison may mix different budgets. Please state the time budget used for each method and, for a fair efficiency comparison, hold the budget constant across methods.
  4. [§3.2.1, §4.2] The UTG exploration depth is set to 1 with no sensitivity analysis. The synthesized functionality table constructed in Section 3.2.1 and the replay guidance that depends on it could plausibly change with exploration depth, and the reported NSR in Table 8 could be sensitive to this choice. The paper should justify the choice of depth and report results for at least one additional depth value to show that the main conclusions are not artifacts of this parameter.
minor comments (5)
  1. [§6] Section 6 labels the structured-output problem as a "threat to validity," but it is a design characteristic of the proposed system; consider presenting it as a limitation rather than a validity threat.
  2. [Table 7] The column header "Input" in Table 7 is ambiguous; it should read "Input value" to match the entity definition in Section 2.3.
  3. [References] The reference list contains two entries for Cooper et al. 2021 with identical titles; one is likely a duplicate or a different paper that needs a distinct title, so please verify and correct.
  4. [§3.1.2] In the prompt template, "restart" and "back" are added to the available actions, but the seven standard actions defined in the same section do not include them; please reconcile the two lists.
  5. [Data availability] The Data availability statement says datasets are available on reasonable request while also pointing to a GitHub repository; please clarify which artifacts are publicly accessible.

Circularity Check

1 steps flagged · score 8.0 of 10

RAG retrieval corpus is built from the same bug-report collections as the 151-report evaluation set, with no reported exclusion of evaluation sentences from the index; this risks converting S2R extraction into label lookup.

  1. self definitional [§3.1.1 Data Collection, §3.1.2 RAG-enhanced S2R extraction, §4.1 Datasets]
    "We begin by crawling a large collection of bug reports from various web resources referenced in prior research (Zhao et al. 2019; Su et al. 2021; Cooper et al. 2021a; Huang et al. 2023). ... we source bug reports from four well-known open-source datasets: (i) the evaluation dataset of ReCDroid (Zhao et al. 2019); (ii) the evaluation dataset of ScopeDroid (Huang et al. 2023); ... (iv) another empirical study on Android bug report reproduction (Su et al. 2021). ..."

    Three of the four RAG database sources (Zhao et al. 2019, Su et al. 2021, Huang et al. 2023) are also three of the four sources of the 151-report evaluation set, and §4.1 only removes duplicate entries within the evaluation set ('we first eliminate duplicate entries'), never stating that evaluation sentences were excluded from the RAG index. Since retrieval in §3.1.2 selects the most cosine-similar sentence and feeds its annotated S2R entities into the prompt, an evaluation sentence present in the index returns its own ground-truth label as the in-context example.

full rationale

The paper's central empirical claim is that RAG-based retrieval of labeled S2R examples substantially improves extraction accuracy and, in turn, bug reproduction success. That claim is only valid if the retrieval index is disjoint from the evaluation data. The paper does not establish this: the RAG database is crawled from the same prior-work collections that later supply the 151-report test set, and no exclusion of test sentences or near-duplicates is reported. Because the RAG prompt includes the retrieved sentence's annotated S2R entities, an exact or near-exact match leaks the ground-truth labels into the prompt. This is a load-bearing circularity for RQ1 and for the NSR improvements that depend on extraction quality. The UTG-based replay component is separate and may contribute independently, but the reported headline gains are not separable from the leakage risk. No self-citation chain, uniqueness import, or ansatz-smuggling issues were found, so the score reflects the dominant data-overlap circularity rather than multiple independent defects.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper's performance claims rest on the quality and non-contamination of the RAG index, the correctness of human labels, and the fairness of the comparison protocol; none of these are verified by shipped artifacts.

free parameters (4)
  • UTG exploration depth = 1
    Set in §4.2 to balance effectiveness and efficiency; not fitted, but a hand-chosen hyperparameter that limits app knowledge.
  • Time budget = 5 minutes
    Set in §4.2 as the termination condition for BugRepro's replay loop; baselines' budgets are not reported.
  • Number of retrieved RAG examples per sentence = 1
    Section 3.1.2 retrieves the most similar sentence for each target sentence. The top-1 choice is a design decision.
  • Available actions list = 7 actions (tap, input, scroll, rotate, delete, double tap, long tap, plus back/restart)
    Defined in §3.1.2 as a fixed action vocabulary; this constrains the extraction output space.
assumptions (4)
  • domain assumption Similar sentences in bug reports imply similar S2R entities.
    The RAG retrieval in §3.1.2 relies on cosine similarity of sentence embeddings to find an example whose S2R entities are transferable to the target sentence. If semantically similar sentences can have different intended actions, extraction accuracy suffers.
  • domain assumption The four-developer labeling process produces correct ground truth.
    The evaluation in §4.4 compares against developer-labeled S2R entities. Labeling errors directly bias all accuracy numbers.
  • domain assumption UI transition graphs capture the app-specific knowledge needed for replay.
    Section 3.2.1 assumes that the UTG built by the UI Automator, with exploration depth 1, is sufficient for the LLM to plan multi-step navigation.
  • domain assumption The evaluation set is representative and the comparison protocol is fair across baselines.
    Section 4.1 filters to 151 text-only reports and no statistical tests are applied; the time budgets for ReCDroid and AdbGPT are not stated, so the fairness of the comparison is unverified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BugRepro: Enhancing Android Bug Reproduction with Domain-Specific Knowledge Integration." pith.science (2026). https://pith.science/paper/OGJXSAAS

@misc{pith2026250514528,
  author       = {Pith},
  title        = {Pith review of: BugRepro: Enhancing Android Bug Reproduction with Domain-Specific Knowledge Integration},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OGJXSAAS}},
  note         = {Machine review of arXiv:2505.14528}
}
read the original abstract

Mobile application development is a fast-paced process where maintaining high-quality user experiences is crucial. Bug reproduction, a key aspect of maintaining app quality, often faces significant challenges. Specifically, when descriptions in bug reports are ambiguous or difficult to comprehend, current approaches fail to extract accurate information. Moreover, modern applications exhibit inherent complexity with multiple pages and diverse functionalities, making it challenging for existing methods to map the relevant information in bug reports to the corresponding UI elements that need to be manipulated. To address these challenges, we propose BugRepro, a novel technique that integrates domain-specific knowledge to enhance the accuracy and efficiency of bug reproduction. BugRepro adopts a Retrieval-Augmented Generation (RAG) approach. It retrieves similar bug reports along with their corresponding steps to reproduce (S2R) entities from an example-rich RAG document. In addition, BugRepro explores the graphical user interface (GUI) of the app and extracts transition graphs from the user interface to incorporate app-specific knowledge to guide large language models (LLMs) in their exploration process. Our experiments demonstrate that BugRepro significantly outperforms two state-of-the-art methods (ReCDroid and AdbGPT). For S2R entity extraction accuracy, it achieves a 7.57 to 28.89 percentage point increase over prior methods. For the bug reproduction success rate, the improvement reaches 74.55% and 152.63%. In reproduction efficiency, the gains are 0.72% and 76.68%.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

59 extracted references · 48 canonical work pages

  1. [1]

    Accessed: 2023-10-05 (2023)

    botherref 42matters : App Store and Google Play Statistics. Accessed: 2023-10-05 (2023). https://42matters.com/stats#available-apps-count botherref

    work page 2023

  2. [2]

    , Koeppel , M

    bchapter Aperdannier , R. , Koeppel , M. , Unger , T. , Schacht , S. , Barkur , S.K. : Systematic evaluation of different approaches on embedding search . In: Future of Information and Communication Conference , pp. 526 -- 536 ( 2024 ). Springer bchapter

    work page 2024

  3. [3]

    https://www.anthropic.com/news/claude-3-5-sonnet

    botherref Anthropic : Claude 3.5 Sonnet. https://www.anthropic.com/news/claude-3-5-sonnet. Accessed: [Insert Date] (2024) botherref

    work page 2024

  4. [4]

    , Venolia , G

    bchapter Aranda , J. , Venolia , G. : The secret life of bugs: Going past the errors and omissions in software repositories . In: 2009 IEEE 31st International Conference on Software Engineering , pp. 298 -- 308 ( 2009 ). IEEE bchapter

    work page 2009

  5. [5]

    , Cooper , N

    bchapter Bernal-C \'a rdenas , C. , Cooper , N. , Moran , K. , Chaparro , O. , Marcus , A. , Poshyvanyk , D. : Translating video recordings of mobile app usages into replayable scenarios . In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering , pp. 309 -- 321 ( 2020 ) bchapter

    work page 2020

  6. [6]

    , Cahyawijaya , S

    botherref Bang , Y. , Cahyawijaya , S. , Lee , N. , Dai , W. , Su , D. , Wilie , B. , Lovenia , H. , Ji , Z. , Yu , T. , Chung , W. , et al.: A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.04023 (2023) botherref

    arXiv 2023

  7. [7]

    , Mann , B

    barticle Brown , T. , Mann , B. , Ryder , N. , Subbiah , M. , Kaplan , J.D. , Dhariwal , P. , Neelakantan , A. , Shyam , P. , Sastry , G. , Askell , A. , : Language models are few-shot learners . Advances in neural information processing systems 33 , 1877 -- 1901 ( 2020 ) barticle

    work page 1901

  8. [8]

    , Bernal-C \'a rdenas , C

    bchapter Cooper , N. , Bernal-C \'a rdenas , C. , Chaparro , O. , Moran , K. , Poshyvanyk , D. : It takes two to tango: Combining visual and textual information for detecting duplicate video-based bug reports . In: 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE) , pp. 957 -- 969 ( 2021 ). IEEE bchapter

    work page 2021

Show all 59 references
  1. [9]

    , Necula , G

    barticle Choi , W. , Necula , G. , Sen , K. : Guided gui testing of android apps with minimal restart and approximate learning . Acm Sigplan Notices 48 ( 10 ), 623 -- 640 ( 2013 ) barticle

    2013

  2. [10]

    , Wang , X

    barticle Chang , Y. , Wang , X. , Wang , J. , Wu , Y. , Yang , L. , Zhu , K. , Chen , H. , Yi , X. , Wang , C. , Wang , Y. , : A survey on evaluation of large language models . ACM Transactions on Intelligent Systems and Technology 15 ( 3 ), 1 -- 45 ( 2024 ) barticle

    2024

  3. [11]

    Accessed: 2023-10-05 (2018)

    botherref Dav : Word2vec. Accessed: 2023-10-05 (2018). https://github.com/dav/word2vec botherref

    2018

  4. [12]

    https://www.deepseek.com/

    botherref DeepSeek : DeepSeek. https://www.deepseek.com/. Accessed: [Insert Date] (2024) botherref

    2024

  5. [13]

    , Kasner , Z

    botherref Du s ek , O. , Kasner , Z. : Evaluating semantic accuracy of data-to-text generation with natural language inference. arXiv preprint arXiv:2011.10819 (2020) botherref

    2020 arXiv

  6. [14]

    , Chen , C

    bchapter Feng , S. , Chen , C. : Gifdroid: Automated replay of visual bug reports for android apps . In: Proceedings of the 44th International Conference on Software Engineering , pp. 1045 -- 1057 ( 2022 ) bchapter

    2022

  7. [15]

    , Chen , C

    bchapter Feng , S. , Chen , C. : Prompting is all you need: Automated android bug replay with large language models . In: Proceedings of the 46th IEEE/ACM International Conference on Software Engineering , pp. 1 -- 13 ( 2024 ) bchapter

    2024

  8. [16]

    , Contributors : AdbGPT: Automating Android Device Interactions with GPT Integration

    botherref Feng , S. , Contributors : AdbGPT: Automating Android Device Interactions with GPT Integration. https://github.com/sidongfeng/AdbGPT. Accessed: 2025-01-06 (2025) botherref

    2025

  9. [17]

    , Prammer , M

    bchapter Fazzini , M. , Prammer , M. , d'Amorim , M. , Orso , A. : Automatically translating bug reports into test cases for mobile apps . In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis , pp. 141 -- 152 ( 2018 ) bchapter

    2018

  10. [18]

    , Ribeiro , L.F

    bchapter Falke , T. , Ribeiro , L.F. , Utama , P.A. , Dagan , I. , Gurevych , I. : Ranking generated summaries by correctness: An interesting but challenging application for natural language inference . In: Proceedings of the 57th Annual Meeting of the Association for Computat...

    2019

  11. [19]

    , Xie , H

    bchapter Fang , S. , Xie , H. , Wang , Y. , Mao , Z. , Zhang , Y. : Read like humans: Autonomous, bidirectional and iterative language modeling for scene text recognition . In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 7098 -- 7107...

    2021

  12. [20]

    , Donos , N

    barticle Galli , C. , Donos , N. , Calciolari , E. : Performance of 4 pre-trained sentence transformer models in the semantic query of a systematic review dataset on peri-implantitis . Information 15 ( 2 ), 68 ( 2024 ) barticle

    2024

  13. [21]

    : BugRepro: Enhancing Android Bug Reproduction with Domain-Specific Knowledge Integration

    botherref Github , A. : BugRepro: Enhancing Android Bug Reproduction with Domain-Specific Knowledge Integration. https://anonymous.4open.science/r/YYyMUSTSTUDY-B889/README.md. Accessed: 2025-04-29 (2025) botherref

    2025

  14. [22]

    , Xiong , Y

    botherref Gao , Y. , Xiong , Y. , Gao , X. , Jia , K. , Pan , J. , Bi , Y. , Dai , Y. , Sun , J. , Wang , H. : Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 (2023) botherref

    2023 arXiv

  15. [23]

    , Montani , I

    barticle Honnibal , M. , Montani , I. : spacy 2: Natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing . To appear 7 ( 1 ), 411 -- 420 ( 2017 ) barticle

    2017

  16. [24]

    , Wang , J

    bchapter Huang , Y. , Wang , J. , Liu , Z. , Wang , S. , Chen , C. , Li , M. , Wang , Q. : Context-aware bug reproduction for mobile apps . In: 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) , pp. 2336 -- 2348 ( 2023 ). IEEE bchapter

    2023

  17. [25]

    , Yu , W

    barticle Huang , L. , Yu , W. , Ma , W. , Zhong , W. , Feng , Z. , Wang , H. , Chen , Q. , Peng , W. , Feng , X. , Qin , B. , : A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions . ACM Transactions on Information Systems 43...

    2025

  18. [26]

    , Jung , S.-g

    barticle Jansen , B.J. , Jung , S.-g. , Salminen , J. : Employing large language models in survey research . Natural Language Processing Journal 4 , 100020 ( 2023 ) barticle

    2023

  19. [27]

    , Lee , N

    barticle Ji , Z. , Lee , N. , Frieske , R. , Yu , T. , Su , D. , Xu , Y. , Ishii , E. , Bang , Y.J. , Madotto , A. , Fung , P. : Survey of hallucination in natural language generation . ACM Computing Surveys 55 ( 12 ), 1 -- 38 ( 2023 ) barticle

    2023

  20. [28]

    , Parnin , C

    bchapter Just , R. , Parnin , C. , Drosos , I. , Ernst , M.D. : Comparing developer-provided to user-provided tests for fault localization and automated program repair . In: Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis , pp. 287 ...

    2018

  21. [29]

    , O g uz , B

    botherref Karpukhin , V. , O g uz , B. , Min , S. , Lewis , P. , Wu , L. , Edunov , S. , Chen , D. , Yih , W.-t. : Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04906 (2020) botherref

    2020 arXiv

  22. [30]

    , Anguelov , D

    botherref Liu , W. , Anguelov , D. , Erhan , D. , Szegedy , C. , Reed , S. , Fu , C. , Berg , A. : Ssd: Single shot multibox detector. in european conference on computer vision. springer (2016) botherref

    2016

  23. [31]

    https://www.llama.com/

    botherref Llama : Llama. https://www.llama.com/. Accessed: [Insert Date] (2024) botherref

    2024

  24. [32]

    u ttler , H. , Lewis , M. , Yih , W.-t. , Rockt \

    barticle Lewis , P. , Perez , E. , Piktus , A. , Petroni , F. , Karpukhin , V. , Goyal , N. , K \"u ttler , H. , Lewis , M. , Yih , W.-t. , Rockt \"a schel , T. , : Retrieval-augmented generation for knowledge-intensive nlp tasks . Advances in Neural Information Processing Sys...

    2020

  25. [33]

    , Yuan , W

    barticle Liu , P. , Yuan , W. , Fu , J. , Jiang , Z. , Hayashi , H. , Neubig , G. : Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing . ACM Computing Surveys 55 ( 9 ), 1 -- 35 ( 2023 ) barticle

    2023

  26. [34]

    , Gong , Y

    botherref Ma , X. , Gong , Y. , He , P. , Zhao , H. , Duan , N. : Query rewriting for retrieval-augmented large language models. arXiv preprint arXiv:2305.14283 (2023) botherref

    2023 arXiv

  27. [35]

    , Iannelli , M

    bchapter Martino , A. , Iannelli , M. , Truong , C. : Knowledge injection to counter large language model (llm) hallucination . In: European Semantic Web Conference , pp. 182 -- 185 ( 2023 ). Springer bchapter

    2023

  28. [36]

    , Sutskever , I

    botherref Mikolov , T. , Sutskever , I. , Chen , K. , Corrado , G.S. , Dean , J. : Distributed representations of words and phrases and their compositionality. Advances in neural information processing systems 26 (2013) botherref

    2013

  29. [37]

    , Cho , K

    botherref Nogueira , R. , Cho , K. : Passage re-ranking with bert. arXiv preprint arXiv:1901.04085 (2019) botherref

    2019 arXiv

  30. [38]

    https://github.com/openatx/uiautomator2

    botherref Netease , Contributors : uiautomator2: Android UI automation framework for Python. https://github.com/openatx/uiautomator2. Accessed: 2025-01-06 (2025) botherref

    2025

  31. [39]

    https://openai.com/index/gpt-4/

    botherref OpenAI : GPT-4. https://openai.com/index/gpt-4/. Accessed: [Insert Date] (2023) botherref

    2023

  32. [40]

    , Contributors : DroidBot: A Lightweight Test Input Generator for Android

    botherref Project , H. , Contributors : DroidBot: A Lightweight Test Input Generator for Android. https://github.com/honeynet/droidbot. Accessed: 2025-01-06 (2025) botherref

    2025

  33. [41]

    , Shin , Y

    barticle Park , Y. , Shin , Y. : Adaptive bi-encoder model selection and ensemble for text classification . Mathematics 12 ( 19 ), 3090 ( 2024 ) barticle

    2024

  34. [42]

    , Gurevych , I

    botherref Reimers , N. , Gurevych , I. : all-MiniLM-L6-v2. Hugging Face (2024). https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2 botherref

    2024

  35. [43]

    , Khomh , F

    barticle Rahman , M.M. , Khomh , F. , Castelluccio , M. : Works for me! cannot reproduce--a large scale empirical study of non-reproducible bugs . Empirical Software Engineering 27 ( 5 ), 111 ( 2022 ) barticle

    2022

  36. [44]

    , Walker , S

    barticle Robertson , S.E. , Walker , S. , Jones , S. , Hancock-Beaulieu , M.M. , Gatford , M. , : Okapi at trec-3 . Nist Special Publication Sp 109 , 109 ( 1995 ) barticle

    1995

  37. [45]

    , Singh , A.K

    botherref Sahoo , P. , Singh , A.K. , Saha , S. , Jain , V. , Mondal , S. , Chadha , A. : A systematic survey of prompt engineering in large language models: Techniques and applications. arXiv preprint arXiv:2402.07927 (2024) botherref

    2024 arXiv

  38. [46]

    , Wang , J

    bchapter Su , T. , Wang , J. , Su , Z. : Benchmarking automated gui testing for android against real-world bugs . In: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , pp. 119 -- 130...

    2021

  39. [47]

    , Yao , W

    botherref Varshney , N. , Yao , W. , Zhang , H. , Chen , J. , Yu , D. : A stitch in time saves nine: Detecting and mitigating hallucinations of llms by validating low-confidence generation. arXiv preprint arXiv:2307.03987 (2023) botherref

    2023 arXiv

  40. [48]

    , Fraser , G

    bchapter White , T.D. , Fraser , G. , Brown , G.J. : Improving random gui testing with image-based widget detection . In: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis , pp. 307 -- 317 ( 2019 ) bchapter

    2019

  41. [49]

    , Sun , J

    bchapter Wendland , T. , Sun , J. , Mahmud , J. , Mansur , S.H. , Huang , S. , Moran , K. , Rubin , J. , Fazzini , M. : Andror2: A dataset of manually-reproduced bug reports for android apps . In: 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR...

    2021

  42. [50]

    , Wang , X

    barticle Wei , J. , Wang , X. , Schuurmans , D. , Bosma , M. , Xia , F. , Chi , E. , Le , Q.V. , Zhou , D. , : Chain-of-thought prompting elicits reasoning in large language models . Advances in neural information processing systems 35 , 24824 -- 24837 ( 2022 ) barticle

    2022

  43. [51]

    , Wu , E

    botherref Wu , K. , Wu , E. , Zou , J. : How faithful are rag models? quantifying the tug-of-war between rag and llms' internal prior. arXiv e-prints, 2404 (2024) botherref

    2024

  44. [52]

    , Yao , Q

    barticle Wang , Y. , Yao , Q. , Kwok , J.T. , Ni , L.M. : Generalizing from a few examples: A survey on few-shot learning . ACM computing surveys (csur) 53 ( 3 ), 1 -- 34 ( 2020 ) barticle

    2020

  45. [53]

    , Zhao , Y

    bchapter Wang , D. , Zhao , Y. , Feng , S. , Zhang , Z. , Halfond , W.G. , Chen , C. , Sun , X. , Shi , J. , Yu , T. : Feedback-driven automated whole bug report reproduction for android apps . In: Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing...

    2024

  46. [54]

    , Fang , H

    bchapter Yang , P. , Fang , H. , Lin , J. : Anserini: Enabling the use of lucene for information retrieval research . In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pp. 1253 -- 1256 ( 2017 ) bchapter

    2017

  47. [55]

    , Su , T

    barticle Zhao , Y. , Su , T. , Liu , Y. , Zheng , W. , Wu , X. , Kavuluru , R. , Halfond , W.G. , Yu , T. : Recdroid+: Automated end-to-end crash reproduction from bug reports for android apps . ACM Transactions on Software Engineering and Methodology (TOSEM) 31 ( 3 ), 1 -- 33...

    2022

  48. [56]

    , Yu , T

    bchapter Zhao , Y. , Yu , T. , Su , T. , Liu , Y. , Zheng , W. , Zhang , J. , Halfond , W.G. : Recdroid: automatically reproducing android application crashes from bug reports . In: 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) , pp. 128 -- 139 ( 2...

    2019

  49. [57]

    , Zhang , A

    botherref Zhang , Z. , Zhang , A. , Li , M. , Smola , A. : Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493 (2022) botherref

    2022 arXiv

  50. [58]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

  51. [59]

    write newline

    " write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.