pith. machine review for the scientific record. sign in

arxiv: 2605.04114 · v1 · submitted 2026-05-05 · 💻 cs.SE · cs.DB

Recognition: 2 theorem links

· Lean Theorem

Semantic Reverse Engineering Legacy Software Applications with ChatGPT, Gemini AI, and Claude AI

Christian Mancas , Diana Christina Mancas
Authors on Pith no claims yet

Pith reviewed 2026-05-08 18:51 UTC · model grok-4.3

classification 💻 cs.SE cs.DB
keywords legacy softwaresemantic reverse engineeringlarge language modelsdatabase applicationsAI code analysissoftware maintenance
0
0 comments X

The pith

ChatGPT, Gemini, and Claude can semantically reverse engineer legacy database applications.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper reports experiments in which three major AI systems are given legacy database code and prompted to produce high-level descriptions of its purpose, data structures, and business rules. The authors test this approach on real old applications to determine what semantic information the models can recover without running the programs. If the technique works reliably, organizations could gain readable explanations of decades-old software whose original developers are no longer available.

Core claim

Large language models supplied with legacy database source code can generate accurate semantic models, entity-relationship descriptions, and functional summaries through targeted prompting.

What carries the argument

Prompting large language models to perform semantic analysis and abstraction on legacy database code.

If this is right

  • Legacy database maintenance can shift from line-by-line reading to reviewing AI-generated semantic summaries.
  • Migration projects gain an automated first pass at understanding the original intent.
  • Documentation gaps in old systems can be filled faster than manual reverse engineering allows.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The method might extend to non-database legacy systems if the prompting patterns generalize.
  • Cross-checking outputs from multiple models could reduce individual hallucinations without added human effort.
  • Over time this could create a library of reusable semantic templates for common legacy patterns.

Load-bearing premise

The AI models can reliably extract accurate semantic understanding from legacy code without significant hallucinations or the need for extensive human correction.

What would settle it

Run the same legacy application through both human experts and the AI pipeline and compare the resulting semantic models for systematic mismatches on data flows or business rules.

Figures

Figures reproduced from arXiv: 2605.04114 by Christian Mancas, Diana Christina Mancas.

Figure 1
Figure 1. Figure 1: MS Access Database Documenter error message displayed when trying to fully document MatBase.mdb view at source ↗
Figure 2
Figure 2. Figure 2: MS Access Database Documenter fragment of a form GUI documentation. Results and Discussion ChatGPT From view at source ↗
Figure 3
Figure 3. Figure 3: MS Access Database Documenter fragment of a form class VBA code documentation view at source ↗
Figure 4
Figure 4. Figure 4: Start of our conversation with ChatGPT (and then Gemini and Claude, just replacing their names) view at source ↗
Figure 5
Figure 5. Figure 5: Start of first ChatGPT answer view at source ↗
Figure 6
Figure 6. Figure 6: Excerpt from the middle of first ChatGPT answer view at source ↗
Figure 7
Figure 7. Figure 7: End of first ChatGPT answer view at source ↗
Figure 8
Figure 8. Figure 8: Our second question view at source ↗
Figure 9
Figure 9. Figure 9: End of ChatGPT’s reply to our second question view at source ↗
Figure 10
Figure 10. Figure 10: Beginning of ChatGPT’s reply to our comment view at source ↗
Figure 11
Figure 11. Figure 11: End of ChatGPT’s reply to our comment. After analyzing our comment, ChatGPT concluded as shown in Figures 12 and 13. As the message shown in view at source ↗
Figure 12
Figure 12. Figure 12: ChatGPT’s verdict on our comment (1 of 2) view at source ↗
Figure 13
Figure 13. Figure 13: ChatGPT’s verdict on our comment (2 of 2) view at source ↗
Figure 14
Figure 14. Figure 14: ChatGPT’s displayed warning after the dialogue shown in Figures 2 to 13 view at source ↗
Figure 15
Figure 15. Figure 15: Second challenge submitted to ChatGPT, Claude, and Gemini (1 of 2). Explanations were right as well, pointing out that the constraint might be violated also when reign and/or parenthood data is up￾dated, as well as when invalid reign periods are stored (because it could not check [29] to see whether the tuple constraint FromY ≤ ToY is enforced too). Its conclusion was right as well: “My guess about your i… view at source ↗
Figure 16
Figure 16. Figure 16: Second challenge submitted to ChatGPT, Claude, and Gemini (2 of 2) view at source ↗
Figure 17
Figure 17. Figure 17: ChatGPT’s second challenge “aerial view” view at source ↗
Figure 18
Figure 18. Figure 18: ChatGPT’s error maintained even after line-by-line code analysis. Then, ChatGPT continued its analysis with the considerations shown in Figures 19 and 20. Next, it qualified its error as being a “subtle slightly drifted decoding” and correctly rewrote the constraint formalization, adding that “My reaction, honestly: Not “I was wrong,” but more precisely: my behavioral summary was slightly mis-scoped; your… view at source ↗
Figure 19
Figure 19. Figure 19: ChatGPT’s analysis of its error (1 of 3) view at source ↗
Figure 20
Figure 20. Figure 20: ChatGPT’s analysis of its error (2 of 3) view at source ↗
Figure 21
Figure 21. Figure 21: ChatGPT’s analysis of its error (3 of 3). We closed our dialog with the remark: “Glad that we are now on the same side of the ocean! Congrats!” ChatGPT replied: “Gladly accepted—and nicely put! Crossing that semantic ocean is exactly what good reverse-engineering is about. And your correction really does put us on the same shoreline, because the distinction you highlighted is one that many people—even exp… view at source ↗
Figure 22
Figure 22. Figure 22: ChatGPT’s closing remarks (1 of 4). Gemini We then asked exactly same questions shown in Figures 4, 8, 15, and 16 to both Gemini and Claude. In its answers, Gemini is very “poetic”, metaphor lover, and extremely verbose; here are some examples (see also [1]): - Think of this as a Bio-Logic “Self-Healing” sensor for your database: it detects an “Ethical Rupture” (a logical mismatch between a City and its C… view at source ↗
Figure 23
Figure 23. Figure 23: ChatGPT’s closing remarks (2 of 4) view at source ↗
Figure 24
Figure 24. Figure 24: ChatGPT’s closing remarks (3 of 4) view at source ↗
Figure 25
Figure 25. Figure 25: ChatGPT’s closing remarks (4 of 4) view at source ↗
Figure 26
Figure 26. Figure 26: Gemini’s first error view at source ↗
Figure 27
Figure 27. Figure 27: Gemini’s second error view at source ↗
Figure 28
Figure 28. Figure 28: Gemini’s second error acknowledgement view at source ↗
Figure 29
Figure 29. Figure 29: Gemini’s second error maintained. Just like reported in [1], rather than seriously engaging in sre Gemini was all the time much more interested in promoting Python, SQLite, and data intelligence apps, proposing us migration from MS technologies and code snippets for, e.g., finding data that violates the two discussed constraints. Claude Claude correctly understood the first “business” rule behind the code… view at source ↗
Figure 30
Figure 30. Figure 30: Claude error when reverse engineering code from Figures 15 and 16 view at source ↗
Figure 31
Figure 31. Figure 31: Claude acknowledgement that it was wrong view at source ↗
read the original abstract

This research paper describes our research results on using ChatGPT, Gemini, and Claude AI to semantically reverse engineer legacy database software applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 0 minor

Summary. The manuscript consists of a single-sentence abstract claiming to describe research results on using ChatGPT, Gemini, and Claude AI to semantically reverse engineer legacy database software applications. No methods, legacy code artifacts, prompts, sample outputs, evaluation metrics, or results sections are present in the full text.

Significance. Application of large language models to semantic reverse engineering of legacy database applications addresses a real industrial need for recovering meaning from undocumented code. If the work had supplied concrete artifacts, reproducible prompts, and correctness assessments, it could have offered a useful case study; the current document supplies none of these elements, so no contribution can be evaluated.

major comments (1)
  1. The full manuscript text contains only the abstract; there is no Methods section, no description of the specific legacy database applications examined, no prompting strategies or workflows, no example AI-generated semantic models, and no evaluation against ground truth. This absence makes the central claim of 'research results' impossible to assess.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their review and for highlighting the gaps in the current manuscript. We agree that the submitted version is incomplete and does not provide sufficient detail to evaluate the claimed research results.

read point-by-point responses

  1. Referee: The full manuscript text contains only the abstract; there is no Methods section, no description of the specific legacy database applications examined, no prompting strategies or workflows, no example AI-generated semantic models, and no evaluation against ground truth. This absence makes the central claim of 'research results' impossible to assess.

    Authors: We acknowledge that the referee is correct: the submitted manuscript consists only of a single-sentence description and lacks all of the elements listed. This was an oversight during submission. In the revised manuscript we will add a complete Methods section, descriptions of the legacy database applications studied, the prompting strategies and workflows used with ChatGPT, Gemini, and Claude, representative AI-generated outputs, and evaluation metrics against ground truth. revision: yes

Circularity Check

0 steps flagged

No circularity: purely descriptive abstract with no derivation chain

full rationale

The manuscript consists solely of a one-sentence abstract stating that it 'describes our research results on using ChatGPT, Gemini, and Claude AI to semantically reverse engineer legacy database software applications.' No equations, models, parameters, predictions, uniqueness theorems, or methodological steps are present. Consequently none of the enumerated circularity patterns (self-definitional, fitted-input-as-prediction, self-citation load-bearing, etc.) can be instantiated, as there is no derivational content whose internal logic could reduce to its own inputs by construction. The document is self-contained as a high-level claim of having performed research; any evaluation of whether that claim is substantiated belongs to correctness or completeness assessment, not circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an empirical report on AI tool application with no mathematical derivations, free parameters, axioms, or invented entities.

pith-pipeline@v0.9.0 · 5298 in / 999 out tokens · 65426 ms · 2026-05-08T18:51:53.230171+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references

  1. [1]

    From Legacy Databases to (Elementary) Mathematical Data Model Schemas with MatBase, ChatGPT , Gemini AI, and Claude AI

    Mancas C. and Mancas DC. “From Legacy Databases to (Elementary) Mathematical Data Model Schemas with MatBase, ChatGPT , Gemini AI, and Claude AI” . Current Trends in Comp. Sci. & App 3.5 (2026): 451-469

    2026

  2. [2]

    ChatGPT (2026)

    OpenAI. ChatGPT (2026). https://chatgpt.com/

    2026

  3. [3]

    Gemini AI (2026)

    Google LLC. Gemini AI (2026)

    2026

  4. [4]

    Claude AI (2026)

    Anthropic. Claude AI (2026). https://claude.ai

    2026

  5. [5]

    MatBase Metadata Catalog Management

    Mancas C. “MatBase Metadata Catalog Management”. Acta Scientific Computer Sciences 2.4 (2020): 25-29. PriMera Scientific Engineering https://primerascientific.com/psen Semantic Reverse Engineering Legacy Software Applications with ChatGPT, Gemini AI, and Claude AI 23

    2020

  6. [6]

    The (Elementary) Mathematical Data Model revisited

    Mancas C. “The (Elementary) Mathematical Data Model revisited” . PriMera Scientific Engineering 5.4 (2024): 78-91

    2024

  7. [7]

    MatBase - A Tool for Transparent Programming while Modeling Data at Conceptual Levels

    Mancas C. “MatBase - A Tool for Transparent Programming while Modeling Data at Conceptual Levels” . In: Proc. 5th Int. Conf. on Comp. Sci. & Inf. Techn. (CSITEC 2019), AIRCC Pub. Corp. Chennai, India (2019): 15-27

    2019

  8. [8]

    Foundations of Databases

    Abiteboul S, Hull R and Vianu V. “Foundations of Databases” . Addison-Wesley, Reading, MA (1995)

    1995

  9. [9]

    A relational model for large shared data banks

    Codd EF. “ A relational model for large shared data banks” . CACM 13.6 (1970): 377-387

    1970

  10. [10]

    Conceptual Data Modeling and Database Design: A Completely Algorithmic Approach. Volume 1: The Shortest Advis able Path

    Mancas C. “Conceptual Data Modeling and Database Design: A Completely Algorithmic Approach. Volume 1: The Shortest Advis able Path” . Apple Academic Press, Waretown, NJ (2015)

    2015

  11. [11]

    The entity-relationship model. Toward a unified view of data

    Chen PP . “The entity-relationship model. Toward a unified view of data” . ACM TODS 1.1 (1976): 9-36

    1976

  12. [12]

    Entity-Relationship Modeling: Foundations of Database Technology

    Thalheim B. “Entity-Relationship Modeling: Foundations of Database Technology” . Springer-Verlag, Berlin, Germany (2000)

    2000

  13. [13]

    On Modelware as the 5th Generation of Programming Languages

    Mancas C. “On Modelware as the 5th Generation of Programming Languages” . Acta Scientific Computer Sciences 2.9 (2020): 24- 26

    2020

  14. [14]

    A Book of Set Theory

    Pinter CC. “ A Book of Set Theory” . Dover Pub. Inc., Mineola, NY (2014)

    2014

  15. [15]

    First-Order Logic. A Concise Introduction. 2nd Edition

    Heil J. “First-Order Logic. A Concise Introduction. 2nd Edition” . Hackett Pub. Co. Inc., Indianapolis / Cambridge, U.S.A (2021)

    2021

  16. [16]

    Temporal Logic and State Systems

    Kroger F. and Merz S. “Temporal Logic and State Systems” . Springer-Verlag, Berlin Heidelberg, Germany (2008)

    2008

  17. [17]

    Should Reverse Engineering Remain a Computer Science Cinderella?

    Mancas C. “Should Reverse Engineering Remain a Computer Science Cinderella?” . J. Inform. Tech. Soft. Eng. S5 (2013): e001

    2013

  18. [18]

    Reverse Engineering and Design Recovery: A Taxonomy

    Chikofsky E. and Cross II J. “Reverse Engineering and Design Recovery: A Taxonomy” . IEEE Softw 7.1 (1190): 13-17

  19. [19]

    The Art of Memory Forensis

    Ligh MH., et al. “The Art of Memory Forensis” . Wiley, Indianapolis, IN (2014)

    2014

  20. [20]

    Reverse Engineering for Beginners

    Yurichev D. “Reverse Engineering for Beginners” . https://bdigital.uvhm.edu.mx/wp-content/uploads/2020/07/Reverse-Engi- neering-for-Beginners.pdf

    2020

  21. [21]

    Rootkits and Bootkits. Reversing Modern Malware and Next Generation Threats

    Matrosov A., et al. “Rootkits and Bootkits. Reversing Modern Malware and Next Generation Threats” . no starch press, San Fran- cisco, CA (2019)

    2019

  22. [22]

    The Ghidra Book, 2nd Edition

    Nance K and Eagle C. “The Ghidra Book, 2nd Edition” . no starch press, San Francisco, CA (2026)

    2026

  23. [23]

    Reversing: Secrets of Reverse Engineering

    Eilam E. “Reversing: Secrets of Reverse Engineering” . Wiley, Indianapolis, IN (2005)

    2005

  24. [24]

    Blue Fox: ARM Assembly Internals & Reverse Engineering

    Markstedter MA. “Blue Fox: ARM Assembly Internals & Reverse Engineering” . Wiley, Hoboken, NJ (2023)

    2023

  25. [25]

    EvilClippy

    outflankln. EvilClippy. https://github.com/outflanknl/EvilClippy/pulls

  26. [26]

    VB Decompiler

    DotFix Software. VB Decompiler. https://www.vb-decompiler.org/products.htm

  27. [27]

    MatBase E-RD Cycles Associated Non-relational Constraints Discovery Assistance Algorithm

    Mancas C. “MatBase E-RD Cycles Associated Non-relational Constraints Discovery Assistance Algorithm” . In: Arai, K. et al. (eds). Intelligent Computing. CompCom 2019. Advances in Intelligent Systems and Computing, Springer, Cham, Switzerland 997 (2019)

    2019

  28. [28]

    On enforcing dyadic-type self-map constraints in MatBase

    Mancas C. “On enforcing dyadic-type self-map constraints in MatBase” . Intl. J. Front. In Eng. & Techn. Res. (IJFETR) 05.01 (2023): 014-026

    2023

  29. [29]

    Design and development of a database software application for managing genealogical trees

    Mancas DC. “Design and development of a database software application for managing genealogical trees” . [Unpublished M.Sc. Dissertation Thesis]. Ovidius University at Constanta, Romania, Mathematics and Informatics Department (2023)

    2023

  30. [30]

    From (Elementary) Mathematical Data Model Schemas to Safe Blazor Web Applications with Claude AI

    Mancas C and Mancas DC. “From (Elementary) Mathematical Data Model Schemas to Safe Blazor Web Applications with Claude AI” . PriMera Scientific Engineering J. 8.4 (2026): 16-37

    2026