REVIEW 4 major objections 7 minor 43 references
Intent-based System Design and Operation
T0 review · 4 major / 7 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read The paper proposes that a high-level 'intent'—what a cloud system must do and how it must behave—can replace most manual design, operation, and improvement work, with LLMs and automation doing the translation.
desk verdict Synthesis paper extending intent-based networking to the cloud lifecycle; no implementation but honest about its verification gap, worth a serious read as a research agenda. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the intent abstraction itself, divided into functional, operational, and refinement intents, connected to a set of four interacting components. Automated design uses a hierarchical generation process—high-level design, low-level implementation, unit tests—extended with end-to-end tests and workload generators. Real-time context awareness models multi-modal runtime data (metrics, logs, traces) as loosely unified event graphs, connects them to domain knowledge (code, docs, runbooks), and produces intent-scoped summaries. Autonomous operation translates operational intent into an ops model that formalizes desired state, observability, risks, and mitigations, feeding executable plans to automation. Continuous improvement couples intent with context to produce refinement intents, resolved by dynamic reconfiguration or by re-running the hierarchical generator until the system satisfies the intent.
What would settle it
A benchmark study in which an LLM-based system is given a realistic functional and operational intent and must produce a deployed multi-service application that meets a stated SLO (for example, p99 latency under load) without any human code or plan modifications; if no current system can pass this on non-trivial intents, the vision remains unvalidated.
Extended reading notes
Core claim
The central claim is that intent can serve as a first-class abstraction for cloud systems, unifying functional and operational requirements into a specification that automation can consume. The paper distinguishes functional intent (features, security, design), operational intent (SLOs, SLAs, monitors, mitigations), and refinement intent (the delta when requirements change). It then proposes that LLMs, guided by these intents and grounded by automation platforms, can generate the system's business logic and tests (automated design), fuse runtime telemetry with code and documentation into a unified context (real-time context awareness), derive an ops model that defines desired state and incident response (autonomous operation), and detect and repair intent violations through reconfiguration or redesign (continuous improvement). The vision is that each phase of the software development lifecycle becomes a translation step from intent rather than a manual activity.
Load-bearing premise
The premise that LLMs can be made reliable enough to generate correct, verifiable system code and operational decisions at cloud scale, despite known hallucinations, weak numerical reasoning, and action-selection inconsistencies.
Editorial extensions
If this is right
- A developer could go from a natural-language description of features and SLOs to a deployed, tested microservice system without hand-writing business logic.
- Operational intent would automatically derive the monitors, alerts, and troubleshooting plans needed to run the service, shortening time-to-mitigation for known incident patterns.
- When behavior deviates from intent—due to bugs, workload shifts, or metastable failures—the system would generate trigger scenarios, redesign candidate systems, and re-test until the intent is satisfied.
- Generated artifacts would arrive with explainability and verification artifacts, so humans can review why the system was built or changed a particular way.
- The same intent can drive both short-term mitigation (reconfiguration) and long-term fixes (redesign), closing the loop between operation and development.
Reading between the lines
- If intent becomes the controlling abstraction, the scarcest skill shifts from coding to expressing precise, verifiable intents; intent authoring may become its own engineering discipline with debugging tools and linters.
- The verification problem—how to guarantee generated code and generated tests are not both wrong—will likely require formal methods integrated into the generation loop, not just more human oversight.
- The framework could generalize beyond cloud to any system with a desired state and observable behavior, such as edge deployments or data pipelines, where intent would absorb environment-specific constraints.
- A testable near-term milestone would be an open benchmark of intents paired with expected SLOs, letting different LLM-based generators be compared on whether deployed systems actually meet the stated intent.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper is a vision/position paper that proposes 'intent' as a high-level abstraction for cloud system design and operation. It argues that users should specify functional and operational intents, and that an LLM-based system should automatically design, implement, operate, and evolve the system. The paper introduces four components — automated design (extending Cerulean), real-time context awareness, system operation via an 'ops model', and continuous improvement — each with an illustrative use case, and closes with a list of future research directions. The central claim is that such a unified abstraction can reduce manual SDLC effort; the paper is explicitly a roadmap rather than a technical evaluation.
Significance. If realized, the vision would be a significant step toward autonomous cloud operations. The paper's strengths are its honest enumeration of challenges (§2.2, §4), its grounding in existing systems (Cerulean, Llexus, Blueprint), and its concrete use cases (microservice generation, incident management, metastable failures). It also explicitly acknowledges the failure mode where generated systems and tests agree but are wrong (§4). However, the paper does not provide a formal definition of intent, a specification of the ops model, or any argument that LLM-generated code and tests can be made reliable at cloud scale. As a position paper it is useful for framing research, but the central claim of holistic automation remains unsupported. The paper would be strengthened by making precise the distinction between the long-term vision and what is claimed to be achievable with current techniques.
major comments (4)
- [§3.1.1 and §4] The verification loop is circular. The implementation and end-to-end tests are generated by the same LLM-based hierarchical process, so passing tests provide weak evidence of correctness; §4 explicitly lists 'cases in which both the generated systems and tests agree, but are wrong' as an open area. Because the paper's central claim is that intent can automate design and operation with reliability, this gap is load-bearing. The paper should either propose an independent oracle (e.g., formal specification, differential testing against a reference implementation, or human-authored invariants) or explicitly disclaim correctness guarantees and state that this is a required research direction. The mere mention of 'robust verification mechanisms' in §2.2 without specifying them is insufficient.
- [§3.3] The 'ops model' is a central component of the proposed architecture (Figure 1), but the paper never defines its representation, semantics, or algorithmic use. The text states that operational intent 'is translated to an operational model' and that the model 'formalizes' observability, risks, and mitigations, yet no concrete structure or example is given. Without this, the claimed ability to 'automate decision-making' (§3.3) is not credible. The paper should at least sketch the model's schema and how it relates to intent and context.
- [§3.2 and §3.3] The paper asserts that real-time context awareness combined with the ops model will 'enable the system to anticipate potential issues, automate decision-making, and execute operational tasks' but provides no account of how context is converted into decisions or how the system handles incorrect or conflicting actions. The 'action selection' challenge in §2.2 is acknowledged, but no mechanism (beyond 'human oversight') is proposed. This is a scalability and correctness concern: LLM action selection at cloud scale with petabyte-scale logs is not addressed. The paper should either present a concrete pipeline or clearly mark these as open problems.
- [§3.1.1 and §3.3.1] The use cases depend on self-cited companion papers (Cerulean [5], Llexus [27], Blueprint [4]) that are listed as 'to appear' or whose details are not included. This prevents the reader from independently assessing whether the claimed extensions are feasible. At minimum, the paper should summarize the relevant results of these systems (e.g., what fraction of generated microservices compiled or passed tests, what success rate Llexus achieves) or cite publicly available versions.
minor comments (7)
- [§1] The phrase 'the system automatically designed, developed, and operated' should be 'designed, developed, and operated automatically'.
- [§3.1.1] There is a typo: 'a automatically generated system' should be 'an automatically generated system'.
- [§3.3 Requirements] The phrase 'to to identify and mitigate these issues' contains a duplicated 'to'.
- [§4] The word 'explanability' is misspelled; it should be 'explainability' in both occurrences.
- [Figure 4] The caption does not explain the elements (Metric, Log, Trace, Docs, Code, Config) or the meaning of the arrows; a short legend would improve readability.
- [§2.1 and §3] The paper calls the approach 'human-in-the-loop' in §2.1 but describes 'autonomous' operation in §3.2 and §3.4; the intended degree of human involvement should be clarified.
- [References and §3.4.1] The metastable failure use case says 'we use the data collected... to generate trigger-scenarios' but does not state whether this generation is LLM-based, and if so, how false positives are controlled.
Circularity Check
No significant circularity: a vision paper with no fitted quantities or derived predictions. The same-LLM implementation/test concern is acknowledged as future work in §4, and the companion self-citations (Cerulean, Llexus, Blueprint) are illustrative building blocks, not load-bearing justifications of the central intent abstraction.
full rationale
This is a position and roadmap paper. It defines 'intent' as a high-level encoding of functional and operational requirements and proposes an architecture (automated design, real-time context awareness, system operation, continuous improvement) to manifest that intent. There are no equations, no fitted parameters, and no empirical predictions whose derivation could reduce to its inputs. The strongest candidate for a circularity concern is the §3.1.1 use case, where the same LLM-based pipeline generates both the microservice implementation and its end-to-end tests from the same functional intent; passing those tests would at best show internal consistency, not genuine correctness. However, the paper itself explicitly flags this limitation in §4, listing 'dealing with the cases in which both the generated systems and tests agree, but are wrong' as a concrete area of future research. Because the paper does not claim that the generated tests constitute a correctness proof, this is an admitted open problem rather than a hidden by-construction result. The manuscript also relies on companion works by overlapping authors, notably Cerulean [5] and Llexus [27], both marked 'To appear', and Blueprint [4]. These are used as illustrative components in specific use cases, and the central vision would not be logically forced even if those tools were replaced by independent alternatives. No uniqueness theorem is imported from the authors' prior work, no ansatz is smuggled in via citation, and no known result is merely renamed. The derivation is therefore self-contained in the sense appropriate to a vision paper: the abstraction and roadmap stand on their own, and the cited companion systems are not load-bearing evidence for the core claim. The minor self-citations and the acknowledged test-verification gap are noted here but do not indicate circularity.
Assumptions & free parameters
assumptions (4)
- domain assumption LLMs can be made reliable enough to produce correct system designs and operations at cloud scale
- domain assumption User intent can be specified with sufficient precision and completeness to drive automated design and operation
- domain assumption Existing automation tools (Kubernetes, OpenTelemetry) can serve as the execution substrate for generated systems
- domain assumption The intent-based networking concept (RFC 9315) is directly transferable to cloud system design and operation
invented entities (2)
-
intent abstraction
-
ops model
Cite this review
Pith. "Pith review of Intent-based System Design and Operation." pith.science (2026). https://pith.science/paper/DYTQOJYU
@misc{pith2026250205984,
author = {Pith},
title = {Pith review of: Intent-based System Design and Operation},
year = {2026},
howpublished = {\url{https://pith.science/paper/DYTQOJYU}},
note = {Machine review of arXiv:2502.05984}
}
read the original abstract
Cloud systems are the backbone of today's computing industry. Yet, these systems remain complicated to design, build, operate, and improve. All these tasks require significant manual effort by both developers and operators of these systems. To reduce this manual burden, in this paper we set forth a vision for achieving holistic automation, intent-based system design and operation. We propose intent as a new abstraction within the context of system design and operation. Intent encodes the functional and operational requirements of the system at a high-level, which can be used to automate design, implementation, operation, and evolution of systems. We detail our vision of intent-based system design, highlight its four key components, and provide a roadmap for the community to enable autonomous systems.
Reference graph
Works this paper leans on
- [5]
-
[27]
P. Las-Casas, A. Kumbhare, R. Fonseca, and S. Agarwal. Llexus: an ai agent system for incident management. SIGOPS Oper. Syst. Rev., 58(1),
- [4]
- [1]
-
[2]
P. Abrahamsson, O. Salo, J. Ronkainen, and J. Warsta. Agile soft- ware development methods: Review and analysis. arXiv preprint arXiv:1709.08439, 2017
arXiv 2017
-
[3]
T. Ahmed, S. Ghosh, C. Bansal, T. Zimmermann, X. Zhang, and S. Ra- jmohan. Recommending root-cause and mitigation steps for cloud incidents using large language models. In 2023 IEEE/ACM 45th Inter- national Conference on Software Engineering (ICSE) , pages 1737–1749. IEEE, 2023
work page 2023
- [6]
-
[7]
How a single chatgpt mistake cost us $10,000+
Asim. How a single chatgpt mistake cost us $10,000+. Accessed 9th June, 2024 from https://web.archive.org/web/20240610032818/https: //asim.bearblog.dev/how-a-single-chatgpt-mistake-cost-us-10000/, 2024
Show all 43 references
-
[8]
Bronson, A
N. Bronson, A. Aghayev, A. Charapko, and T. Zhu. Metastable failures in distributed systems. In Proceedings of the Workshop on Hot Topics in Operating Systems, pages 221–227, 2021
2021
-
[9]
Y. Chen, H. Xie, M. Ma, Y. Kang, X. Gao, L. Shi, Y. Cao, X. Gao, H. Fan, M. Wen, et al. Empowering practical root cause analysis by large language models for cloud incidents. arXiv preprint arXiv:2305.15778, 2023
2023 arXiv
-
[10]
Clemm, L
A. Clemm, L. Ciavaglia, L. Z. Granville, and J. Tantsura. Rfc 9315: Intent-based networking - concepts and definitions, 2022
2022
-
[11]
Cockcroft
A. Cockcroft. The evolution of microservices. (April 2016). Retrieved October 2020 from https://www.slideshare.net/adriancockcroft/ evolution-of-microservices-craft-conference, 2016
2016
-
[12]
Cockcroft
A. Cockcroft. Microservices workshop: Why, what, and how to get there. (April 2016). Retrieved October 2020 from https://www.slideshare.net/adriancockcroft/microservices- workshop-craft-conference, 2016
2016
-
[13]
Ganatra, A
V. Ganatra, A. Parayil, S. Ghosh, Y. Kang, M. Ma, C. Bansal, S. Nath, and J. Mace. Detection is better than cure: A cloud incidents perspective. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineeri...
1902
-
[14]
Ghemawat, R
S. Ghemawat, R. Grandl, S. Petrovic, M. Whittaker, P. Patel, I. Posva, and A. Vahdat. Towards modern development of cloud applications. In Proceedings of the 19th Workshop on Hot Topics in Operating Systems , pages 110–117, 2023
2023
-
[15]
Ghosh, M
S. Ghosh, M. Shetty, C. Bansal, and S. Nath. How to fight production incidents? an empirical study on a large-scale cloud service. In Pro- ceedings of the 13th Symposium on Cloud Computing , pages 126–141, 2022
2022
-
[16]
A. Gluck. Introducing domain-oriented microservice architecture. Accessed June 2024 from https://www.uber.com/blog/microservice- architecture/, 2020
2024
-
[17]
D. Goel, F. Husain, A. Singh, S. Ghosh, A. Parayil, C. Bansal, X. Zhang, and S. Rajmohan. X-lifecycle learning for cloud incident management using llms. arXiv preprint arXiv:2404.03662, 2024
2024 arXiv
-
[18]
E. Haddad. Service-oriented architecture: Scaling the uber engineering codebase as we grow. (September 2015). Retrieved October 2020 from https://eng.uber.com/service-oriented-architecture/, 2015
2015
-
[19]
Hamadanian, B
P. Hamadanian, B. Arzani, S. Fouladi, S. K. R. Kakarla, R. Fonseca, D. Billor, A. Cheema, E. Nkposong, and R. Chandra. A holistic view of ai-driven network incident management. In Proceedings of the 22nd ACM Workshop on Hot Topics in Networks , pages 180–188, 2023
2023
-
[20]
M. Hashemi. The infrastructure behind twitter : Scale. (January 2017). Retrieved February 2021 from https://blog.twitter.com/engineering/ en_us/topics/infrastructure/2017/the-infrastructure-behind-twitter- scale.html, 2017
2017
-
[21]
Huang, M
L. Huang, M. Magnusson, A. B. Muralikrishna, S. Estyak, R. Isaacs, A. Aghayev, T. Zhu, and A. Charapko. Metastable failures in the wild. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages 73–90, 2022
2022
-
[22]
Huang, W
L. Huang, W. Yu, W. Ma, W. Zhong, Z. Feng, H. Wang, Q. Chen, W. Peng, X. Feng, B. Qin, et al. A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions. arXiv preprint arXiv:2311.05232, 2023
2023 arXiv
-
[23]
Jiang, C
Y. Jiang, C. Zhang, S. He, Z. Yang, M. Ma, S. Qin, Y. Kang, Y. Dang, S. Rajmohan, Q. Lin, et al. Xpert: Empowering incident management with query recommendations via large language models.arXiv preprint arXiv:2312.11988, 2023. Intent-based System Design and Operation Anand et al
2023 arXiv
-
[24]
Kaldor, J
J. Kaldor, J. Mace, M. Bejda, E. Gao, W. Kuropatwa, J. O’Neill, K. W. Ong, B. Schaller, P. Shan, B. Viscomi, et al. Canopy: An end-to-end performance tracing and analysis system. In Proceedings of the 26th symposium on operating systems principles , pages 34–50, 2017
2017
-
[25]
B. Lampson. Hints and principles for computer system design. arXiv preprint arXiv:2011.02455, 2020
2011 arXiv
-
[26]
B. W. Lampson. Hints for computer system design. In Proceedings of the ninth ACM symposium on Operating systems principles , pages 33–48, 1983
1983
-
[28]
C. Lee, T. Yang, Z. Chen, Y. Su, and M. Lyu. Maat: Performance metric anomaly anticipation for cloud services with conditional diffusion. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 116–128. IEEE, 2023
2023
-
[29]
Y. Li, X. Zhang, S. He, Z. Chen, Y. Kang, J. Liu, L. Li, Y. Dang, F. Gao, Z. Xu, et al. An intelligent framework for timely, accurate, and com- prehensive cloud incident detection. ACM SIGOPS Operating Systems Review, 56(1):1–7, 2022
2022
-
[30]
F. Lin, K. Muzumdar, N. P. Laptev, M.-V. Curelea, S. Lee, and S. Sankar. Fast dimensional analysis for root cause investigation in a large-scale service environment. Proceedings of the ACM on Measurement and Analysis of Computing Systems, 4(2):1–23, 2020
2020
-
[31]
J. Liu, J. Zhu, S. He, P. He, Z. Zheng, and M. R. Lyu. Logzip: Extracting hidden structures via iterative clustering for log compression. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 863–873. IEEE, 2019
2019
-
[32]
J. C. Mogul. Emergent (mis) behavior vs. complex software systems. ACM SIGOPS Operating Systems Review , 40(4):293–304, 2006
2006
-
[33]
C. M. Rosenberg and L. Moonen. Spectrum-based log diagnosis. In Proceedings of the 14th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) , pages 1–12, 2020
2020
-
[34]
D. Roy, X. Zhang, R. Bhave, C. Bansal, P. Las-Casas, R. Fonseca, and S. Rajmohan. Exploring llm-based agents for root cause analysis. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering , FSE 2024, page 208–219, New York...
2024
-
[35]
Seshagiri, S
V. Seshagiri, S. Balyan, V. Anand, K. Dhole, I. Sharma, A. Wildani, J. Cambronero, and A. Züfle. Chatting with logs: An exploratory study on finetuning llms for logql. arXiv preprint arXiv:2412.03612, 2024
2024 arXiv
-
[36]
E. Shanks. Kubernetes - desired state and control loops. Ac- cessed July, 2024 from https://theithollow.com/2019/09/16/kubernetes- desired-state-and-control-loops/, 2019
2024
-
[37]
Somashekar, K
G. Somashekar, K. Tandon, A. Kini, C.-C. Chang, P. Husak, R. Bhagwan, M. Das, A. Gandhi, and N. Natarajan. OPPerTune: Post-Deployment configuration tuning of services made easy. In21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24) , pages 1101–1120,...
2024
-
[38]
Srinivas, F
P. Srinivas, F. Husain, A. Parayil, A. Choure, C. Bansal, and S. Rajmohan. Intelligent monitoring framework for cloud services: A data-driven ap- proach. In Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice , pages 381–391, 2024
2024
-
[39]
H. Wang, G. K. Tangirala, G. P. Naidu, C. Mayville, A. Roy, J. Sun, and R. B. Mandava. Anomaly detection for incident response at scale. arXiv preprint arXiv:2404.16887, 2024
2024 arXiv
-
[40]
Z. Xie, Y. Zheng, L. Ottens, K. Zhang, C. Kozyrakis, and J. Mace. Cloud atlas: Efficient fault localization for clou systems using lan- guage models and causal insight. Accessed 11th July, 2024 from https://people.mpi-sws.org/~jcmace/papers/xie2024cloud.pdf, 2024
2024
-
[41]
G. Yu, P. Chen, Y. Li, H. Chen, X. Li, and Z. Zheng. Nezha: Interpretable fine-grained root causes analysis for microservices on multi-modal observability data. In Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of So...
2023
-
[42]
Zhang, X
D. Zhang, X. Zhang, C. Bansal, P. Las-Casas, R. Fonseca, and S. Rajmo- han. Lm-pace: Confidence estimation by large language models for effective root causing of cloud incidents. In Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software E...
2024
-
[43]
Zhang, S
X. Zhang, S. Ghosh, C. Bansal, R. Wang, M. Ma, Y. Kang, and S. Raj- mohan. Automated root causing of cloud incidents using in-context learning with gpt-4. arXiv preprint arXiv:2401.13810, 2024
2024 arXiv
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.