REVIEW 4 major objections 5 minor 1 cited by
LLM-Generated Microservice Implementations from RESTful API Definitions
T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read This paper claims a multi-agent LLM system can generate and debug a Dockerized CRUD microservice from a natural-language API description.
desk verdict A clear tool report for an LLM multi-agent pipeline whose central iteration-reduction claim is unsupported by the small perception-based evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is a function-calling LLM loop organized around a JSON file-map representation of the whole server codebase: each file path is a key and the file's content is the value, so agents can generate, validate, clean, persist, and patch entire projects without any single agent holding a full repository in context. The second load-bearing piece is local Docker execution. Instead of trusting an LLM's abstract self-check, the code-tester agent runs docker-compose up, fetches real container logs, and issues curl requests, so the code-fixer agent works from observed runtime behavior. The named agents are spec-generator, code-generator, json-cleaner, code-tester, and code-fixer, each with a small set of functions it can call in the user's environment.
What would settle it
Run the pipeline on a fixed set of, say, fifty CRUD prompts, inject a known runtime fault into each generated service, and count how often the code-tester's log analysis names the true fault and the code-fixer's patch makes a fresh test suite pass without manual help. If fewer than half of the injected faults are fixed end-to-end, the central claim that log-driven repair reduces iterations is unsupported.
Extended reading notes
Core claim
The central claim is that a modular set of LLM agents, each with a narrow job and access to the user's local environment through function calling, can take a natural-language API description through the full CRUD microservice development loop. The spec-generator agent drafts an OpenAPI contract; the code-generator and json-cleaner agents render it into a complete Express.js server as a JSON object whose keys are file paths and whose values are file contents; the code-tester agent starts the service with Docker Compose, sends HTTP requests to it, and reads container logs; and the code-fixer agent rewrites the saved files in the same JSON form when logs expose a problem. The system is scoped to CRUD and REST, uses GPT-4 as the underlying model, and is intended for prototyping and MVP development rather than complex enterprise customizations. The surveyed practitioners' ratings and comments are presented as indicative evidence that the workflow is faster and more integrated than manual coding, not as proof of production readiness.
Load-bearing premise
The whole loop depends on the LLM correctly interpreting execution logs and error messages and generating fixes that actually address the reported fault; if log analysis misdiagnoses too often, the promised reduction in iterations and manual debugging collapses.
Editorial extensions
If this is right
- If the pipeline works as described, a developer can go from a sentence like 'product service with add/edit/delete/list operations' to a running Dockerized Express.js service with business logic through one interface.
- The feedback loop grounds debugging advice in the real execution environment: logs from the user's local containers drive the fixes, so environment-specific issues can be caught before handoff.
- Because each agent holds only its own context and communicates through files and functions, swapping the code-generator agent could adapt the system to another language or framework without rebuilding the whole pipeline.
- If adopted for prototyping, the approach could move API-contract design earlier in the lifecycle, since the OpenAPI spec is produced and reviewed before any server code is written.
- The paper's own limitations imply the pipeline's benefit is strongest for small CRUD services; complex authentication, validation, and custom business logic still need manual work.
Reading between the lines
- A direct extension of the agent design would be to swap GPT-4 for open-weight models; if the agent roles and the JSON file-map handoff are the real source of reliability, quality should degrade gracefully rather than collapse.
- The same JSON-chunk representation suggests a scaling path, which the paper names as future work but does not test: split a larger codebase into chunks handled by coordinated fixer agents.
- A controlled before/after study with identical CRUD tasks, counting iterations to a passing service and time on task, would test the time-saving claim more directly than the survey's self-reported ratings.
- The local Docker-and-log loop could generalize beyond code generation to a general 'run, observe, repair' pattern for LLM-generated infrastructure scripts, where execution feedback replaces static validation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents a multi-agent LLM-based system that automates API-first development of RESTful microservices. From a natural-language description of a CRUD service, the system generates an OpenAPI specification (spec-generator agent), produces Express.js server code with business logic in a predefined folder structure (code-generator and JSON-cleaner agents), runs the service in Docker, and then enters a log-driven repair loop in which a code-tester agent reads execution logs and a code-fixer agent proposes and applies fixes. The authors report survey results from six industry practitioners, with thematic analysis indicating perceived time savings, automation of repetitive tasks, and reduced tool switching. The paper claims in the abstract that the log-analysis feedback loop reduces the number of iterations required to produce functional and robust services, but the evaluation does not measure iterations, repair success rates, or objective code correctness. The code and survey data are made publicly available.
Significance. If the system works as described, it is a practical contribution to rapid prototyping of small CRUD microservices: the multi-agent architecture with local Docker execution and log-fed repair is more ambitious than static code generation, and the public availability of code and data (arXiv/GitHub/Zenodo) is a genuine strength. However, the significance is currently limited by the evaluation: the only evidence is a six-person convenience-sample survey with self-reported ratings and thematic coding. The paper's strongest claims about reduced iterations and robust services are not measured, and the authors themselves concede in Section 6 that the suggested fixes are sometimes insufficient and require user intervention. The contribution is therefore best understood as a tool demonstration with preliminary usability feedback, not as a validated claim of improved development efficiency or reliability.
major comments (4)
- [Abstract, §3.2.5, §3.3, §4] The paper's headline claim that the log-driven repair loop 'reduces the number of iterations required to produce functional and robust services' is never operationalized or measured. Section 3.3 and Section 4 report only six participants' subjective ratings and thematic analysis from Table 3; there is no record of how many repair cycles each generated service required, no success rate for the code-fixer agent, and no baseline comparison (e.g., manual development or a version of the system without the feedback loop). Section 6 concedes that 'in some situations, the fixes suggested by the agents are not sufficient to fully address the problem, and additional manual adjustments are needed.' Without objective convergence data, the central value proposition is unsupported. Please add quantitative measurements: per-service iteration counts, the proportion of services that pass a defined automated test suite without manual intervention, and a comparison condition.
- [§3.1, §3.3.2, §4] The research questions ask about comparative properties (RQ1: 'compared to traditional manual coding methods'; RQ2: 'reduce the need for switching'; RQ3: 'reducing manual coding efforts'), but the evaluation design cannot answer comparative questions. The six-participant convenience sample includes four practitioners with no microservice experience, and the quantitative evidence consists of means and modes of Likert-scale self-assessments (e.g., Table 3 Q3 and Q8). The paper acknowledges in Section 4 that 'due to the small sample size, these results should be interpreted as indicative rather than definitive,' but the conclusions in Section 5 and Section 6 are worded more strongly than this caveat supports. Either narrow the conclusions to perceived usability, or add a controlled baseline with objective measures such as task completion time, number of tool switches, and completion of a defined functional task.
- [§3.2.5, §5] The load-bearing assumption that LLM log analysis accurately diagnoses faults and generates correct fixes is not tested. Section 5 states that 'the accuracy of error detection and fix suggestions depends on the LLM's understanding of logs and its ability to interpret deployment-specific issues.' The paper provides no injection-based experiment (e.g., introducing known defects into generated code and measuring whether the loop converges to the correct patch) and no comparison against an alternative repair strategy, such as static analysis or a baseline LLM without log access. Because the claimed advantage over tools like GitHub Copilot rests on using execution logs to drive fixes, this mechanism must be assessed directly rather than through subjective ratings of 'debugging effectiveness.'
- [§3.2, §3.2.4, §3.3.1] The claim that the system produces 'fully functional API code, including business logic' and 'robust services' is not verified by any objective functional test. The survey's closest item (Table 3 Q5) asks whether the tool's testing features 'effectively validate' functionality, which is again a subjective rating. The described validation in Section 3.2.4 consists of users asking natural-language requests such as 'get the list of products' and observing responses; this is not a systematic test suite. To support the functionality and robustness claims, the authors should run the generated services through a contract-based automated test suite derived from the OpenAPI specification (e.g., end-to-end HTTP tests for each endpoint and error case) and report pass rates.
minor comments (5)
- [§4.1, §4.2] Several cross-references appear as 'table ??' instead of pointing to Table 3; these should be corrected.
- [Figure 4 caption] The caption reads 'Intracting With Server'; this should be 'Interacting with Server.'
- [References] Dudjak and Martinović (2020) appears twice in the reference list with essentially the same bibliographic information; one entry should be removed.
- [References] The references for Chauhan (2024a) and Chauhan (2024b) list only titles, without URLs or DOIs, despite the paper's emphasis on public availability of code and data; please include the GitHub and Zenodo links in the reference entries.
- [§2.3] The inline URL contains spaces ('https://github.com/sirbh/sample generated cpi/blob/main/openapi spec.yml'); it should be formatted as a proper link and verified to resolve.
Circularity Check
No circular derivation; the paper is an empirical tool/usability report whose claims rest on a described workflow and survey feedback, not on a self-referential derivation.
full rationale
This paper is an empirical systems and usability report rather than a derivation: it describes a multi-agent LLM pipeline (OpenAPI specification generation, server code generation, Docker validation, and log-based repair) and evaluates it with a six-practitioner survey. There is no formal derivation chain, no fitted parameter whose output is defined in terms of its input, and no equation reused as a prediction. The abstract's claim that log analysis reduces the number of iterations needed to produce functional services is not directly measured, and Section 6 concedes that agent fixes are sometimes insufficient and user intervention is necessary; however, this is a threat to empirical support or evidence quality, not circularity. The Rasheed et al. self-citations (e.g., 2024a CodePori comparison and 2024b-d background) and Chauhan (2024a/b) data/instructions links are contextual and do not carry the load of the central claim: the system's behavior is characterized by its described workflow and by participant perceptions, not by a self-cited theorem or imported uniqueness argument. No equation is reused, no parameter fitted to a subset is renamed as a prediction, and no ansatz is smuggled in via citation. Consequently, no specific circular step can be exhibited, and the appropriate finding is no significant circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption The user's local environment has Docker and docker-compose available and can run the generated Express.js services.
- domain assumption GPT-4's function-calling feature reliably executes commands, saves files, and returns usable results.
- domain assumption The code-generator agent's JSON output can be cleaned and parsed without losing content.
- domain assumption Self-reported ratings on a 1-10 scale are valid indicators of actual usability and effectiveness.
Cite this review
Pith. "Pith review of LLM-Generated Microservice Implementations from RESTful API Definitions." pith.science (2026). https://pith.science/paper/QZ776B2L
@misc{pith2026250209766,
author = {Pith},
title = {Pith review of: LLM-Generated Microservice Implementations from RESTful API Definitions},
year = {2026},
howpublished = {\url{https://pith.science/paper/QZ776B2L}},
note = {Machine review of arXiv:2502.09766}
}
read the original abstract
The growing need for scalable, maintainable, and fast-deploying systems has made microservice architecture widely popular in software development. This paper presents a system that uses Large Language Models (LLMs) to automate the API-first development of RESTful microservices. This system assists in creating OpenAPI specification, generating server code from it, and refining the code through a feedback loop that analyzes execution logs and error messages. By focusing on the API-first methodology, this system ensures that microservices are designed with well-defined interfaces, promoting consistency and reliability across the development life-cycle. The integration of log analysis enables the LLM to detect and address issues efficiently, reducing the number of iterations required to produce functional and robust services. This process automates the generation of microservices and also simplifies the debugging and refinement phases, allowing developers to focus on higher-level design and integration tasks. This system has the potential to benefit software developers, architects, and organizations to speed up software development cycles and reducing manual effort. To assess the potential of the system, we conducted surveys with six industry practitioners. After surveying practitioners, the system demonstrated notable advantages in enhancing development speed, automating repetitive tasks, and simplifying the prototyping process. While experienced developers appreciated its efficiency for specific tasks, some expressed concerns about its limitations in handling advanced customizations and larger scale projects. The code is publicly available at https://github.com/sirbh/code-gen
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
Vibe Coding vs. Agentic Coding: Fundamentals and Practical Implications of Agentic AI
A qualitative taxonomy positions vibe coding and agentic coding as complementary paradigms rather than rivals in AI-assisted software development.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...
-
[2]
Adam, S. I., Moedjahedy, J. H., and Maramis, J. (2020). Restful web service implementation on unklab information system using json web token (jwt). In 2020 2nd International Conference on Cybernetics and Intelligent System (ICORIS) , pages 1--6
work page 2020
-
[3]
Alkaissi, H. and McFarlane, S. I. (2023). Artificial hallucinations in chatgpt: implications in scientific writing. Cureus , 15(2)
work page 2023
-
[4]
Beaulieu, N., Dascalu, S. M., and Hand, E. (2022). Api-first design: a survey of the state of academia and industry. In ITNG 2022 19th International Conference on Information Technology-New Generations , pages 73--79. Springer
work page 2022
-
[5]
Boyatzis, R. E. (1998). Transforming qualitative information: Thematic analysis and code development . Sage
work page 1998
-
[6]
Casas, S., Cruz, D., Vidal, G., and Constanzo, M. (2021). Uses and applications of the openapi/swagger specification: a systematic mapping of the literature. In 2021 40th International Conference of the Chilean Computer Science Society (SCCC) , pages 1--8
work page 2021
-
[7]
Chauhan, S. (2024a). Code for service generation and instructions
work page 2024
-
[8]
Chauhan, S. (2024b). Llm-generated microservice implementations from restful api definitions
work page 2024
Show all 40 references
-
[9]
Chen, R., Li, S., and Li, Z. (2017). From monolith to microservices: A dataflow-driven approach. In 2017 24th Asia-Pacific Software Engineering Conference (APSEC) , pages 466--475
2017
-
[10]
Chen, X., Liao, Y., and Yu, W. (2024). Generative ai in higher art education. In 2024 6th International Conference on Computer Science and Technologies in Education (CSTE) , pages 135--140
2024
-
[11]
F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D
Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. (2017). Deep reinforcement learning from human preferences. Advances in neural information processing systems , 30
2017
-
[12]
De, B. (2023). API Management : An Architect's Guide to Developing and Managing APIs for Your Organization . Apress, Berkeley, CA, 2nd ed. 2023. edition
2023
-
[13]
and Raman, V
Ding, S. and Raman, V. (2024). Harness the power of generative ai in healthcare with amazon ai/ml services. In 2024 IEEE 12th International Conference on Healthcare Informatics (ICHI) , pages 490--492
2024
-
[14]
and Martinovi \'c , G
Dudjak, M. and Martinovi \'c , G. (2020). An api-first methodology for designing a microservice-based backend as a service platform. Information Technology and Control , 49(2):206--223
2020
-
[15]
and Martinović, G
Dudjak, M. and Martinović, G. (2020). An api-first methodology for designing a microservice-based backend as a service platform. Information technology and control , 49(2):206--223
2020
-
[16]
Jiang, J., Wang, F., Shen, J., Kim, S., and Kim, S. (2024). A survey on large language models for code generation. arXiv preprint arXiv:2406.00515
2024 arXiv
-
[17]
C., Tielman, M
Jorge, C. C., Tielman, M. L., and Jonker, C. M. (2022). Artificial trust as a tool in human-ai teams. In 2022 17th ACM/IEEE International Conference on Human-Robot Interaction (HRI) , pages 1155--1157
2022
-
[18]
S., Dhatterwal, J
Kaswan, K. S., Dhatterwal, J. S., Malik, K., and Baliyan, A. (2023). Generative ai: A review on models and applications. In 2023 International Conference on Communication, Security and Artificial Intelligence (ICCSAI) , pages 699--704
2023
-
[19]
W., Keutzer, K., and Gholami, A
Kim, S., Moon, S., Tabrizi, R., Lee, N., Mahoney, M. W., Keutzer, K., and Gholami, A. (2024). An llm compiler for parallel function calling
2024
-
[20]
Lazar, K., Vetzler, M., Uziel, G., Boaz, D., Goldbraich, E., Amid, D., and Anaby-Tavor, A. (2024). Specrawler: Generating openapi specifications from api documentation using large language models
2024
-
[21]
Lercher, A. (2024). Managing api evolution in microservice architecture. In 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) , pages 195--197
2024
-
[22]
S., Wang, Y., and Zhang, L
Liu, J., Xia, C. S., Wang, Y., and Zhang, L. (2023). Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation
2023
-
[23]
and Nadi, S
Nguyen, N. and Nadi, S. (2022). An empirical evaluation of github copilot's code suggestions. In Proceedings of the 19th International Conference on Mining Software Repositories , pages 1--5
2022
-
[24]
Olson, L. (2024). Custom developer gpt for ethical ai solutions. In 2024 IEEE/ACM 3rd International Conference on AI Engineering – Software Engineering for AI (CAIN) , pages 282--283
2024
-
[25]
Ponelat, J. S. and Rosenstock, L. L. (2022). Designing APIs with Swagger and OpenAPI . Simon and Schuster
2022
-
[26]
A., Kemell, K.-K., Waseem, M., Saari, M., Systä, K., and Abrahamsson, P
Rasheed, Z., Sami, M. A., Kemell, K.-K., Waseem, M., Saari, M., Systä, K., and Abrahamsson, P. (2024a). Codepori: Large-scale system for autonomous software development using multi-agent technology
2024
-
[27]
A., Rasku, J., Kemell, K.-K., Zhang, Z., Harjamaki, J., Siddeeq, S., Lahti, S., Herda, T., Nurminen, M., et al
Rasheed, Z., Sami, M. A., Rasku, J., Kemell, K.-K., Zhang, Z., Harjamaki, J., Siddeeq, S., Lahti, S., Herda, T., Nurminen, M., et al. (2024b). Timeless: A vision for the next generation of software development. arXiv preprint arXiv:2411.08507
2024 arXiv
-
[28]
A., Waseem, M., Kemell, K.-K., Wang, X., Nguyen, A., Syst \"a , K., and Abrahamsson, P
Rasheed, Z., Sami, M. A., Waseem, M., Kemell, K.-K., Wang, X., Nguyen, A., Syst \"a , K., and Abrahamsson, P. (2024c). Ai-powered code review with llms: Early results. arXiv preprint arXiv:2404.18496
2024
-
[29]
N., and Abrahamsson, P
Rasheed, Z., Waseem, M., Ahmad, A., Kemell, K.-K., Xiaofeng, W., Duc, A. N., and Abrahamsson, P. (2024d). Can large language models serve as data analysts? a multi-agent assisted approach for qualitative data analysis. arXiv preprint arXiv:2402.01386
2024
-
[30]
K., Ahmad, A., Sami, M
Rasheed, Z., Waseem, M., Kemell, K. K., Ahmad, A., Sami, M. A., Rasku, J., Syst \"a , K., and Abrahamsson, P. (2025). Large language models for code generation: The practitioners perspective. arXiv preprint arXiv:2501.16998
2025 arXiv
-
[31]
N., Syst \"a , K., and Abrahamsson, P
Rasheed, Z., Waseem, M., Kemell, K.-K., Xiaofeng, W., Duc, A. N., Syst \"a , K., and Abrahamsson, P. (2023). Autonomous agents in software development: A vision paper. arXiv preprint arXiv:2311.18440
2023 arXiv
-
[32]
M., Heil, S., Grigera, J., Gaedke, M., and Rossi, G
Rivero, J. M., Heil, S., Grigera, J., Gaedke, M., and Rossi, G. (2013). Mockapi: An agile approach supporting api-first web application development. In Daniel, F., Dolog, P., and Li, Q., editors, Web Engineering . Springer Berlin Heidelberg
2013
-
[33]
Romani, Y., Tibermacine, O., and Tibermacine, C. (2022). Towards migrating legacy software systems to microservice-based architectures: a data-centric process for microservice identification. In 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C...
2022
-
[34]
Sallam, M. (2023). Chatgpt utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. In Healthcare , volume 11, page 887. MDPI
2023
-
[35]
and Bhowmik, B
Saxena, D. and Bhowmik, B. (2023). Paradigm shift from monolithic to microservices. In 2023 IEEE International Conference on Recent Advances in Systems Science and Engineering (RASSE) , pages 1--7
2023
-
[36]
Wang, J., Cao, L., Luo, X., Zhou, Z., Xie, J., Jatowt, A., and Cai, Y. (2023). Enhancing large language models for secure code generation: A dataset-driven study on vulnerability mitigation
2023
-
[37]
Wang, J., Yang, Q., and Chen, Y. (2024). A large language model–based approach for automatically optimizing bim. In 2024 43rd Chinese Control Conference (CCC) , pages 8518--8523
2024
-
[38]
Zhang, B., Liang, P., Zhou, X., Ahmad, A., and Waseem, M. (2023). Practices and challenges of using github copilot: An empirical study. In Proceedings of the 35th International Conference on Software Engineering and Knowledge Engineering , volume 2023 of SEKE2023 , page 124–12...
2023
-
[39]
Zhao, W. X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., Dong, Z., Du, Y., Yang, C., Chen, Y., Chen, Z., Jiang, J., Ren, R., Li, Y., Tang, X., Liu, Z., Liu, P., Nie, J.-Y., and Wen, J.-R. (2024). A survey of large language models
2024
-
[40]
and Koziolek, A
Zimmermann, D. and Koziolek, A. (2023). Gui-based software testing: An automated approach using gpt-4 and selenium webdriver. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW) , pages 171--174
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.