Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

PersonaBOT: Bringing Customer Personas to Life with LLMs and RAG

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Synthetic personas from LLM prompting raised a construction-equipment chatbot's accuracy rating from 5.88 to 6.42.

desk verdict Useful applied case study, but the headline accuracy claim is confounded and the McNemar reporting is internally inconsistent; worth a serious referee if revisions are pursued. read the letter →

arxiv 2505.17156 v1 pith:4B2E6RYO submitted 2025-05-22 cs.CL cs.LG

Muhammed Rizwan , Lars Carlsson , Mohammad Loni This is my paper
classification cs.CLcs.LG
keywords customerpersonaslargelanguagemodelsretrieval-augmentedgenerationfew-shotpromptingchain-of-thoughtMcNemartestsyntheticbusinessdecisionsupport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper reports on a proof-of-concept system that turns customer success stories into synthetic personas using two prompting strategies, then feeds them into a retrieval-augmented generation (RAG) chatbot so business users can query persona data. Its central claim is that the augmentation works: adding the synthetic personas and segment-specific information raised the chatbot's average accuracy rating from 5.88 to 6.42 on a 10-point scale in user evaluations, and 81.82% of participants rated the updated system at least somewhat useful. A secondary claim is that few-shot prompting produces more complete personas than chain-of-thought prompting, while chain-of-thought is faster and uses fewer tokens. If correct, the paper shows a practical, low-cost path from qualitative customer narratives to interactive persona-based decision support in an industrial setting.

What carries the argument

The load-bearing mechanism is a RAG loop whose knowledge base is itself the experimental variable. Verified personas, synthetic personas, and segment background text are embedded and indexed; a hybrid retrieval step (keyword, semantic, and vector search with HNSW approximate nearest neighbors) returns the top three documents, and a language model generates answers from those documents plus a system prompt. Persona quality is scored by three expert evaluators on completeness, relevance, and consistency using binary yes/no judgments analyzed with McNemar's test for paired nominal data; the test statistic uses only discordant pairs, $\chi^2 = (b-c)^2/(b+c)$. The same evaluation form is repeated before and after knowledge-base augmentation, making the augmentation the manipulated factor.

What would settle it

Run a controlled A/B test with the same participant pool, the same system prompt, and only the knowledge base changed between rounds; if the average accuracy rating does not move from roughly 5.88 toward 6.42 in that setting, the paper's central claim that augmenting with synthetic personas improves accuracy would be refuted.

Watch

Extended reading notes

Core claim

On its own terms, the work establishes that synthetic personas can be generated from publicly available customer success stories and integrated with verified personas in a RAG chatbot without degrading, and slightly improving, perceived answer accuracy. The authors compare few-shot and chain-of-thought prompting for persona generation: few-shot produces statistically significantly more complete personas (McNemar test with p=0.0063), while the two methods do not differ significantly on relevance or consistency. Because chain-of-thought is substantially more efficient (average 2.79 seconds and 2064 tokens versus 3.66 seconds and 3506 tokens), the authors select chain-of-thought personas for the augmented knowledge base. After augmentation, the average accuracy rating rises from 5.88 to 6.42, complex-query handling improves (no "never" responses), and 81.82% of the 12 evaluators rate the system at least somewhat useful.

Load-bearing premise

The comparison that carries the headline result assumes the before and after evaluations are comparable, even though the first had 8 participants and the second had 12, and the system prompt was revised between rounds.

Editorial extensions

If this is right

  • Organizations can use few-shot prompting when completeness of a persona matters most, because it captured significantly more key details from source stories than chain-of-thought.
  • When response time and token cost dominate, chain-of-thought is the cheaper generation route without statistically significant losses in relevance or consistency.
  • A RAG chatbot that already works with verified personas can be augmented with synthetic ones and extra segment text, and users perceive the augmented version as more accurate and at least somewhat useful.
  • Persona-based RAG appears feasible as a lightweight decision-support layer for business functions like marketing, R&D, and customer relations, with the caveat that the study is a single-company proof of concept.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A controlled replication that holds the system prompt fixed would isolate the knowledge-base effect from the prompt revision, which the current design leaves confounded.
  • Because the persona evaluation used only three evaluators, the McNemar significance on completeness rests on small discordant counts; a replication with more evaluators would show whether few-shot's advantage is stable.
  • The same pipeline could be extended to other qualitative corpora, such as support tickets, survey responses, or forum posts, where the main cost is prompt design and chunking rather than manual persona interviews.
  • An automated retrieval metric, such as answer-groundedness or retrieval recall, run alongside user ratings would let future deployments track augmentation quality without recruiting new evaluators each round.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper reports a design-science study at Volvo Construction Equipment in which a RAG chatbot is built on verified customer personas, synthetic personas are generated from customer success stories using Few-Shot and Chain-of-Thought prompting, the two prompting methods are compared on completeness, relevance, and consistency with McNemar's test, and the chatbot's knowledge base is then augmented with the better-performing synthetic personas and additional segment information. The claimed findings are that Few-Shot significantly outperforms CoT on completeness, CoT is more efficient in time and tokens, and augmenting the knowledge base raised the average accuracy rating from 5.88 to 6.42 out of 10, with 81.82% of evaluators finding the updated system at least somewhat useful.

Significance. The study addresses a practical gap in applying LLM-generated personas in an industrial RAG setting, and it provides a concrete, real-data case study with a structured evaluation. The artifact description is detailed enough to be replicable, and the use of a paired statistical test for persona comparison is appropriate. If the results hold after correction, the work offers useful evidence on the prompt-engineering trade-off between completeness and efficiency, and on the incremental value of knowledge-base augmentation in a business chatbot. However, the statistical reporting contains an internal inconsistency, and the before/after chatbot evaluation is confounded by simultaneous changes to participant group, system prompt, and knowledge base, which substantially weakens the central empirical claims in their current form.

major comments (3)
  1. [§4.2.1, Table 5] The reported McNemar test statistic of 1.0 is inconsistent with the contingency table shown. For b=11 and c=1, the chi-square statistic is (11-1)^2/(11+1) = 8.33, not 1.0. The reported p-value of 0.0063 matches the exact binomial McNemar test (two-sided), which does not produce a chi-square statistic of 1.0. Please correct the statistic and explicitly state which form of the McNemar test was used, or report the exact binomial p-value without a chi-square statistic.
  2. [§3.8 vs. §4.1 and §4.3] The claim that the same participant group was used in the initial and augmented chatbot evaluations is contradicted by the reported numbers: §4.1 states 8 stakeholders in the initial evaluation, while §4.3 states 12 stakeholders in the post-augmentation evaluation. Additionally, §3.8 states that the system prompt was revised at the same time as the knowledge-base augmentation. Consequently, the observed increase from 5.88 to 6.42 cannot be attributed to the knowledge-base augmentation alone. Please report the actual overlap between participant groups, describe the prompt change, and either provide a matched comparison or explicitly reframe the result as a pilot observation with multiple simultaneous changes.
  3. [§4.3.1 and §4.3.2] The usefulness percentage is internally inconsistent. The text in §4.3.1 reports ratings from 12 participants: 7 'somewhat', 2 'mostly', 1 'perfectly', 1 'not at all', and 1 'not well', which gives 10 out of 12 (83.3%) rating the system at least 'somewhat useful', not 81.82%. If the denominator is 11 because one response was missing, that should be stated. In addition, no significance test, confidence interval, or effect size is provided for the 5.88-to-6.42 average-accuracy change, so the improvement is well within plausible sampling noise for 8 vs. 12 unpaired ratings.
minor comments (5)
  1. [§3.7.2, Table 3] The consistency metric is phrased as 'Does the persona add any incorrect or made-up information that is not in the customer success story?' A 'Yes' answer is therefore a negative outcome, but the paper's analysis in §4.2 treats 'Yes' as the favorable category without clarifying the coding direction. Please state how responses were mapped for analysis.
  2. [§4.2.1, Table 8] The summary table for prompting techniques leaves the quality-metric cells for CoT empty, and the labels for Few-Shot are 'Statistically significant' / 'Statistically insignificant' without reporting the actual p-values or effect direction. Fill in the complete contingency table summaries and p-values so readers can verify the claims.
  3. [§4.2.2] The paper selects CoT for the final system because of its efficiency, even though Few-Shot was found to be statistically better on completeness. This is a permissible design trade-off, but it should be justified more explicitly given the study's stated goal of generating complete personas; otherwise the choice appears to contradict the evaluation outcome.
  4. [§4.1.1] The sentence 'When evaluating the potential of the system to reduce the workforce' should read 'reduce the workload', based on the surrounding text and Table 2.
  5. [§3.5.2] The reference 'Figure 3.5.2' is incorrect; it should be a numbered figure reference such as 'Figure 3' or 'Figure 4', and the figure itself is not clearly labeled in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: persona evaluation is externally anchored; the pre/post chatbot comparison is confounded but not circular.

full rationale

The claimed derivation chain is self-contained and not circular. Synthetic personas are produced from customer success stories via GPT-4o Mini with few-shot or CoT prompts; the persona evaluation asks expert raters whether each generated persona captures details of the same source story ('Does the persona include all the important details (like role, challenges, expectations etc.) from the customer success story to fully understand the customer?'), so the evaluation criterion is external to the generation process and is not defined in terms of the model output. The chatbot improvement claim is a measured pre/post comparison: Section 3.8 states that 'the same evaluation method, participant group, and questionnaire from the initial evaluation were used again,' and the reported averages 5.88 and 6.42 are observed ratings, not fitted parameters renamed as predictions. No equation in the paper reduces one claimed result to an input quantity, and no uniqueness theorem or ansatz is imported from the authors' prior work. The only author self-citation ([25], Loni et al.) supports a generic statement about LLM text-generation applications and is not load-bearing. The strongest concern is a validity confound, not circularity: Section 3.8 says the participant group was reused while Section 4.1 reports 8 stakeholders and Section 4.3 reports 12, and the system prompt was simultaneously revised; therefore the 0.54-point accuracy increase cannot be cleanly attributed to knowledge-base augmentation. That is an internal-validity limitation, consistent with the paper's own Section 6.2 caveats about subjective, small-sample evaluation, but it does not make the derivation circular.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claims rest on domain assumptions about statistical correctness, comparability of evaluation rounds, and data representativeness. No parameters are fitted to the outcome, and no new entities are postulated. The assumptions are listed because they are load-bearing for the conclusions.

assumptions (5)
  • domain assumption The McNemar test is correctly applied and its reported statistic matches the contingency table.
    The paper relies on this test to conclude Few-Shot outperforms CoT on completeness, but the reported statistic (1.0) is inconsistent with the b=11, c=1 table.
  • domain assumption The two chatbot evaluations (initial n=8, augmented n=12) are comparable despite different participant groups and a revised system prompt.
    The paper attributes the accuracy increase from 5.88 to 6.42 to knowledge base augmentation, but confounds make this attribution unsupported.
  • domain assumption Customer success stories provide sufficient and unbiased information to generate accurate customer personas.
    The paper itself notes in Section 6.2 that success stories emphasize positive outcomes, so generated personas may lack a balanced perspective.
  • domain assumption The random subset of five stories is representative of the 24 stories for evaluating prompting methods.
    The completeness comparison rests on ratings of only five stories by three evaluators.
  • domain assumption GPT-4o Mini's generated personas are stable and replicable enough to support the conclusions.
    No seed, temperature, or decoding parameters are reported; LLM outputs can be stochastic.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PersonaBOT: Bringing Customer Personas to Life with LLMs and RAG." pith.science (2026). https://pith.science/paper/4B2E6RYO

@misc{pith2026250517156,
  author       = {Pith},
  title        = {Pith review of: PersonaBOT: Bringing Customer Personas to Life with LLMs and RAG},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4B2E6RYO}},
  note         = {Machine review of arXiv:2505.17156}
}
read the original abstract

The introduction of Large Language Models (LLMs) has significantly transformed Natural Language Processing (NLP) applications by enabling more advanced analysis of customer personas. At Volvo Construction Equipment (VCE), customer personas have traditionally been developed through qualitative methods, which are time-consuming and lack scalability. The main objective of this paper is to generate synthetic customer personas and integrate them into a Retrieval-Augmented Generation (RAG) chatbot to support decision-making in business processes. To this end, we first focus on developing a persona-based RAG chatbot integrated with verified personas. Next, synthetic personas are generated using Few-Shot and Chain-of-Thought (CoT) prompting techniques and evaluated based on completeness, relevance, and consistency using McNemar's test. In the final step, the chatbot's knowledge base is augmented with synthetic personas and additional segment information to assess improvements in response accuracy and practical utility. Key findings indicate that Few-Shot prompting outperformed CoT in generating more complete personas, while CoT demonstrated greater efficiency in terms of response time and token usage. After augmenting the knowledge base, the average accuracy rating of the chatbot increased from 5.88 to 6.42 on a 10-point scale, and 81.82% of participants found the updated system useful in business contexts.

Figures

Figures reproduced from arXiv: 2505.17156 by the authors.

Figure 1
Figure 1. Research Method 3.3 Data Preparation This section discusses the process involved in the data preparation. Each subsection describes the process involved for that specific data type. 3.3.1 Customer Success Story The first type of data used was the Customer Success Stories, which served as input for generating synthetic customer personas. To extract these stories, web scraping was employed using the Python library Bea… view at source ↗
Figure 2
Figure 2. Example Synthetic Persona - Few Shot [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Example Synthetic Persona - Cot 3.5 Building the RAG system The role of the RAG system is to act as a conversational agent that allows users to query information based on customer persona data and general information about different segments. The system consists of two main components: (1) Retrieval Component is responsible for storing, indexing, and retrieving relevant documents; and (2) Generation Component that i… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Persona Generation Process 3.5.1 Retrieval Component The retrieval component was built using Azure AI Search4 that acts as a dedicated search engine and storage of data. The implementation process involved in building this component included the following steps: 1. Cre…
Figure 3.5
Figure 3.5. Figure 3.5: 2 shows the overall process of the chatbot system, highlighting the flow from data indexing and retrieval to [PITH_FULL_IMAGE:figures/full_fig_p011_3_5.png]
Figure 5
Figure 5. Figure 5: Distribution of the accuracy rating The ability of the chatbot to interpret and respond to complex queries such as providing details on customer persona was measured using a Likert scale. The majority of the participants indicated that the chatbot responded correctly "…
Figure 6
Figure 6. Figure 6: Ability of the system to provide response to complex query [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: Ability of the system to provide clear and concise response [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Alignment of system in business need [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: Workload reduction and improvement in automation [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Comparison of the metrics- Completeness Relevance: Relevance was assessed to determine whether the persona focused only on the important and relevant details from the source material. The test yielded a p-value of 0.6250, suggesting that there is no significant differ…
Figure 11
Figure 11. Figure 11: Average Time Taken 4.2.2 Summary of Findings The evaluation revealed that Few-Shot prompting significantly outperformed CoT prompting in terms of completeness. Evaluators generally rated Few-Shot personas as more complete than those generated using CoT. For the other …
Figure 12
Figure 12. Figure 12: Average Tokens Consumed 4.3 Results for Research Question 3: Impact of Knowledge Base Augmentation This section presents the results after augmenting the chatbot’s knowledge base with synthetic personas and segment￾specific information. 12 stakeholders from business f…
Figure 13
Figure 13. Figure 13: Distribution of the accuracy rating - post improvements [PITH_FULL_IMAGE:figures/full_fig_p020_13.png]
Figure 14
Figure 14. Figure 14: Ability of the system to provide response to complex query - post augmentation [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Alignment of system in business need - post updation [PITH_FULL_IMAGE:figures/full_fig_p021_15.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. AlterAtlas: Shifting Travel Planning from AI Generation to Validation via Persona-Driven Simulations

    cs.HC 2026-07 conditional novelty 6.0 of 10

    AlterAtlas replaces one-shot AI itinerary generation with an interactive validation loop where persona-driven simulations expose route-level constraints and guide iterative revision.

Reference graph

Works this paper leans on

39 extracted references · 29 canonical work pages · cited by 1 Pith paper

  1. [1]

    In The Quantum AI Era of Neuromarketing, pp

    More, Pratik and Pothula, Shiva Sai Kiran (2025).Quantum Leap in Customer Persona Development: Enhancing Consumer Profiles and Experiences Using Quantum AI. In The Quantum AI Era of Neuromarketing, pp. 133–156

    work page 2025

  2. [2]

    Digital Persona: Reflection on the Power of Generative AI for Customer Profiling in Social Media Marketing

    Morandé (2023). Digital Persona: Reflection on the Power of Generative AI for Customer Profiling in Social Media Marketing

    work page 2023

  3. [3]

    Journal of Retailing and Consumer Services, 81, 103975

    Praveen, SV and Gajjar, Pranshav and Ray, Rajeev Kumar and Dutt, Ashutosh (2024).Crafting clarity: Lever- aging large language models to decode consumer reviews . Journal of Retailing and Consumer Services, 81, 103975

    work page 2024

  4. [4]

    LLMs’ ways of seeing User Personas

    Panda, Swaroop (2024). LLMs’ ways of seeing User Personas. arXiv preprint arXiv:2409.14858

    arXiv 2024

  5. [5]

    arXiv preprint arXiv:2411.00027

    Zhang, Zhehao and Rossi, Ryan A and Kveton, Branislav and Shao, Yijia and Yang, Diyi and Zamani, Hamed and Dernoncourt, Franck and Barrow, Joe and Yu, Tong and Kim, Sungchul and others (2024).Personalization of large language models: A survey. arXiv preprint arXiv:2411.00027

    arXiv 2024

  6. [6]

    arXiv preprint arXiv:2406.01171

    Tseng, Yu-Min and Huang, Yu-Chao and Hsiao, Teng-Yun and Hsu, Yu-Ching and Foo, Jia-Yin and Huang, Chao- Wei and Chen, Yun-Nung (2024).Two tales of persona in llms: A survey of role-playing and personalization. arXiv preprint arXiv:2406.01171

    arXiv 2024

  7. [7]

    CloChat: Under- standing How People Customize, Interact, and Experience Personas in Large Language Models

    Ha, Juhye and Jeon, Hyeon and Han, Daeun and Seo, Jinwook and Oh, Changhoon (2024). CloChat: Under- standing How People Customize, Interact, and Experience Personas in Large Language Models. In Proceedings of the CHI Conference on Human Factors in Computing Systems, pp. 1–24

    work page 2024

  8. [8]

    Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models

    Cheng, Myra and Durmus, Esin and Jurafsky, Dan (2023). Marked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models. arXiv preprint arXiv:2305.18189

    arXiv 2023

Show all 39 references
  1. [9]

    Automating Customer Needs Analysis: A Comparative Study of Large Language Models in the Travel Industry

    Barandoni, Simone and Chiarello, Filippo and Cascone, Lorenzo and Marrale, Emiliano and Puccio, Salvatore (2024). Automating Customer Needs Analysis: A Comparative Study of Large Language Models in the Travel Industry. arXiv preprint arXiv:2404.17975

    2024 arXiv

  2. [10]

    PersonaLLM: Investigating the Ability of Large Language Models to Express Big Five Personality Traits

    Jiang, Hang and Zhang, Xiajie and Cao, Xubo and Kabbara, Jad (2023). PersonaLLM: Investigating the Ability of Large Language Models to Express Big Five Personality Traits. arXiv preprint arXiv, 2305

    2023

  3. [11]

    Lewis, Patrick and Perez, Ethan and Piktus, Aleksandra and Petroni, Fabio and Karpukhin, Vladimir and Goyal, Naman and K. (2020). Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Systems, 33, 9459–9474

    2020

  4. [12]

    Persona-L has Entered the Chat: Leveraging LLM and Ability-based Framework for Personas of People with Complex Needs

    Sun, Lipeipei and Qin, Tianzi and Hu, Anran and Zhang, Jiale and Lin, Shuojia and Chen, Jianyan and Ali, Mona and Prpa, Mirjana (2024). Persona-L has Entered the Chat: Leveraging LLM and Ability-based Framework for Personas of People with Complex Needs. arXiv preprint arXiv:2409.15604

    2024 arXiv

  5. [13]

    Data-driven persona development

    McGinn, Jennifer and Kotamraju, Nalini (2008). Data-driven persona development . In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 1521–1524

    2008

  6. [14]

    Data-driven personas: Constructing archetypal users with clickstreams and user telemetry

    Zhang, Xiang and Brown, Hans-Frederick and Shankar, Anil (2016). Data-driven personas: Constructing archetypal users with clickstreams and user telemetry. In Proceedings of the 2016 CHI conference on human factors in computing systems, pp. 5350–5359

    2016

  7. [15]

    Automatic persona generation (APG) a rationale and demonstration

    Jung, Soon-gyo and Salminen, Joni and Kwak, Haewoon and An, Jisun and Jansen, Bernard J (2018). Automatic persona generation (APG) a rationale and demonstration. In Proceedings of the 2018 conference on human information interaction & retrieval, pp. 321–324

    2018

  8. [16]

    SOMONITOR: Combining Explainable AI & Large Language Models for Marketing Analytics

    Farseev, Aleksandr and Yang, Qi and Ongpin, Marlo and Gossoudarev, Ilia and Chu-Farseeva, Yu-Yi and Nikolenko, Sergey (2024). SOMONITOR: Combining Explainable AI & Large Language Models for Marketing Analytics. arXiv e-prints, arXiv–2407. 23 PersonaBOT

    2024

  9. [17]

    Salminen, Joni and Liu, Chang and Pian, Wenjing and Chi, Jianxing and H. (2024). Deus ex machina and personas from large language models: investigating the composition of AI-generated persona descriptions. In Proceedings of the 2024 CHI Conference on Human Factors in Computing...

    2024

  10. [18]

    Personagen: A tool for generating personas from user feedback

    Zhang, Xishuo and Liu, Lin and Wang, Yi and Liu, Xiao and Wang, Hailong and Ren, Anqi and Arora, Chetan (2023). Personagen: A tool for generating personas from user feedback . In 2023 IEEE 31st International Requirements Engineering Conference (RE), pp. 353–354

    2023

  11. [19]

    arXiv preprint arXiv:2310.06391

    De Paoli, Stefano (2023).Improved prompting and process for writing user personas with LLMs, using qualitative interviews: Capturing behaviour and personality traits of users. arXiv preprint arXiv:2310.06391

    2023 arXiv

  12. [20]

    In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work, pp

    Goel, Toshali and Shaer, Orit and Delcourt, Catherine and Gu, Quan and Cooper, Angel (2023).Preparing future designers for human-ai collaboration in persona creation. In Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work, pp. 1–14

    2023

  13. [21]

    LLMs: A Comprehensive Survey of Applications, Challenges, Datasets, Models, Limitations, and Future Prospects

    Hadi, Muhammad Usman and Al-Tashi, Qasem and Qureshi, Rizwan and Shah, Abbas and Muneer, Amgad and Irfan, Muhammad and Zafar, Anas and Shaikh, Muhammad Bilal and Akhtar, Naveed and Al-Garadi, Mohammed Ali and others (n.d.). LLMs: A Comprehensive Survey of Applications, Challen...

  14. [22]

    A Reality check of the benefits of LLM in business

    Cheung, Ming (2024). A Reality check of the benefits of LLM in business. arXiv preprint arXiv:2406.10249

    2024 arXiv

  15. [23]

    Prompt Engineering – Enhance Results with Prompt Engineering Strategies

    OpenAI (2025). Prompt Engineering – Enhance Results with Prompt Engineering Strategies

    2025

  16. [24]

    Azure AI Search: Outperforming Vector Search with Hybrid Retrieval and Reranking

    Microsoft Azure Team (2024). Azure AI Search: Outperforming Vector Search with Hybrid Retrieval and Reranking

    2024

  17. [25]

    A Review on Generative AI Models for Synthetic Medical Text, Time Series, and Longitudinal Data

    Loni, Mohammad and Poursalim, Fatemeh and Asadi, Mehdi and Gharehbaghi, Arash (2024). A Review on Generative AI Models for Synthetic Medical Text, Time Series, and Longitudinal Data . arXiv preprint arXiv:2411.12274

    2024 arXiv

  18. [26]

    A method framework for design science research

    Johannesson, Paul and Perjons, Erik and Johannesson, Paul and Perjons, Erik (2014). A method framework for design science research. An introduction to design science, 75–89

    2014

  19. [27]

    Effective use of the McNemar test

    Pembury Smith, Matilda QR and Ruxton, Graeme D (2020). Effective use of the McNemar test . Behavioral Ecology and Sociobiology, 74, 1–9

    2020

  20. [28]

    Language models are few-shot learners

    Brown, Tom and Mann, Benjamin and Ryder, Nick and Subbiah, Melanie and Kaplan, Jared D and Dhariwal, Prafulla and Neelakantan, Arvind and Shyam, Pranav and Sastry, Girish and Askell, Amanda and others (2020). Language models are few-shot learners. Advances in neural informatio...

    2020

  21. [29]

    Chain-of-thought prompting elicits reasoning in large language models

    Wei, Jason and Wang, Xuezhi and Schuurmans, Dale and Bosma, Maarten and Xia, Fei and Chi, Ed and Le, Quoc V and Zhou, Denny and others (2022). Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35, 24824–24837

    2022

  22. [30]

    Gpt-4 technical report

    Achiam, Josh and Adler, Steven and Agarwal, Sandhini and Ahmad, Lama and Akkaya, Ilge and Aleman, Florencia Leoni and Almeida, Diogo and Altenschmidt, Janko and Altman, Sam and Anadkat, Shyamal and others (2023). Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    2023 arXiv

  23. [31]

    Generative adversarial nets

    Goodfellow, Ian J and Pouget-Abadie, Jean and Mirza, Mehdi and Xu, Bing and Warde-Farley, David and Ozair, Sherjil and Courville, Aaron and Bengio, Yoshua (2014). Generative adversarial nets. Advances in neural information processing systems, 27

    2014

  24. [32]

    Auto-encoding variational bayes

    Kingma, Diederik P and Welling, Max and others (2013). Auto-encoding variational bayes. Banff, Canada

    2013

  25. [33]

    Latent dirichlet allocation

    Blei, David M and Ng, Andrew Y and Jordan, Michael I (2003). Latent dirichlet allocation. Journal of machine Learning research, 3(Jan), 993–1022

    2003

  26. [34]

    Structural topic models for open-ended survey responses

    Roberts, Margaret E and Stewart, Brandon M and Tingley, Dustin and Lucas, Christopher and Leder-Luis, Jetson and Gadarian, Shana Kushner and Albertson, Bethany and Rand, David G (2014). Structural topic models for open-ended survey responses. American journal of political scie...

    2014

  27. [35]

    (2007).A design science research methodology for information systems research

    Peffers, Ken and Tuunanen, Tuure and Rothenberger, Marcus and Chatterjee, S. (2007).A design science research methodology for information systems research. Journal of Management Information Systems, 24, 45-77

    2007

  28. [36]

    Journal of management information systems, 24(3), 45–77

    Peffers, Ken and Tuunanen, Tuure and Rothenberger, Marcus A and Chatterjee, Samir (2007).A design science research methodology for information systems research. Journal of management information systems, 24(3), 45–77

    2007

  29. [37]

    Retrieval- augmented generation with graphs (graphrag)

    Han, Haoyu and Wang, Yu and Shomer, Harry and Guo, Kai and Ding, Jiayuan and Lei, Yongjia and Halappanavar, Mahantesh and Rossi, Ryan A and Mukherjee, Subhabrata and Tang, Xianfeng and others (2024). Retrieval- augmented generation with graphs (graphrag). arXiv preprint arXiv:...

    2024 arXiv

  30. [38]

    arXiv preprint arXiv:2401.15391

    Tang, Yixuan and Yang, Yi (2024).Multihop-rag: Benchmarking retrieval-augmented generation for multi-hop queries. arXiv preprint arXiv:2401.15391

    2024 arXiv

  31. [39]

    Schreiber, William, White, Jules, and Schmidt, Douglas C. (2024). A Pattern Language for Persona-based Interactions with LLMs. 25

    2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.