REVIEW 3 major objections 5 minor 21 references
Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows
T0 review · 3 major / 5 minor · reviewed 2026-07-10 · glm-5.2
Pith's one-line read Treat the workflow itself as knowledge, not just a process that produces it
desk verdict Solid conceptual vocabulary for LLM workflow persistence, but the operational gap between 'semantic persistence' and provenance-plus-checkpointing is not demonstrated read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The derive/infer boundary: derive is deterministic computation over available state; infer is mediated LLM judgment under declared context and executor-controlled capability policy. The executor mediates all model output, validating it before it can influence a declared branch or trigger an external effect. Semantic objects persist in a shared knowledge substrate with typed roles, stable identity, and queryability across execution boundaries.
What would settle it
Build two systems with identical workflows: one using the proposed semantic object model with pre-declared roles, and one using a standard checkpoint-and-trace system with post-hoc provenance export. If reviewers performing inspection, attribution, and audit tasks show no measurable improvement in accuracy or speed when using the semantic model, the core claim—that pre-declared semantic persistence adds value beyond execution persistence—is not supported.
Extended reading notes
Core claim
The paper introduces a semantic object schema in which workflow artifacts are assigned typed roles (workflow-definition, workflow-instance, inference-record, context-snapshot, approval-record, panel-record, dependency-link, and others) before execution begins, rather than being captured as after-the-fact provenance. The load-bearing mechanism is the derive/infer boundary: derive denotes deterministic, replayable computation over available workflow state, while infer denotes LLM-mediated judgment that requires declared context, a prompt, an expected return type, validation, persistence, and an explicit capability policy. The LLM fills declared infer slots but never directly controls workflow—
Load-bearing premise
The paper assumes that assigning declared semantic roles to workflow artifacts before execution provides meaningful benefits over provenance systems that capture the same information after the fact. If pre-declaration offers no practical advantage over post-hoc provenance, the model reduces to a vocabulary overlay on existing persistence mechanisms.
Editorial extensions
If this is right
- If workflows persist as typed knowledge objects, then queries like 'show me every decision that depended on this document' or 'list all deferred deliberations' become substrate-level operations rather than custom provenance engineering.
- The derive/infer split creates an auditable seam: every point where an LLM exercised judgment is explicitly marked, context-bounded, and linked to its downstream effects, making influence chains inspectable rather than hidden in execution traces.
- Workflow instances as persistent objects enable semantic resumption: a prior panel decision or inference result can be reused when compatibility conditions are met, without re-running the entire workflow.
- The model provides a vocabulary for distinguishing human authorization (approval) from structured deliberation (panel), giving review and escalation paths explicit semantic status rather than treating them as generic 'human steps.'
Reading between the lines
- If the model were implemented, a natural test would be whether review tasks (attribution, audit, reproducibility) are measurably faster or more accurate when artifacts have pre-declared semantic roles versus when the same information is reconstructed from post-hoc provenance logs.
- The model's treatment of the LLM as a 'mediated peripheral' rather than an autonomous agent suggests a design pattern where multiple LLM services could fill different infer slots in the same workflow under different capability policies, with the executor enforcing isolation and recording cross-inference dependencies.
- The supersession-link concept (revised or disputed objects linked to their predecessors) could support a form of knowledge evolution tracking where changed premises trigger re-derivation of dependent objects while preserving the full history of judgment changes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a conceptual model in which LLM-mediated workflow definitions, running instances, inference records, context snapshots, and dependency relations are represented as persistent, typed knowledge objects in a shared 'knowledge substrate.' The central semantic distinction is between derive (deterministic computation over available state) and infer (mediated LLM judgment under declared context and executor-controlled capability policy). The model is Lisp-inspired but language-independent, drawing on live-image and symbolic-environment traditions. The paper defines a vocabulary of semantic object kinds (Table 1), operating primitives (Table 2), and provides a worked example (Appendix B) and a preliminary PROV-DM mapping (Appendix C). The authors are explicit that this is a conceptual proposal: there is no formal transition semantics, no implementation, and no empirical evaluation.
Significance. The paper addresses a genuine representational gap in current LLM workflow systems. While frameworks such as LangGraph, AgentSPEX, and DSPy make control flow explicit and provide checkpointing, they generally treat intermediate artifacts as execution traces rather than first-class knowledge objects with pre-declared semantic roles. The derive/infer distinction is a useful conceptual contribution: it forces workflow authors to make explicit which steps involve nondeterministic model judgment and which are deterministic, replayable computation. The panel/approval separation (Section 3.3, Table 1) is a thoughtful refinement that distinguishes authorization gates from structured deliberation. The preliminary PROV-DM mapping in Appendix C demonstrates provenance-compatibility. However, the paper's significance is currently limited by the absence of a concrete demonstration that the proposed semantic object model enables capabilities beyond what structured provenance plus checkpointing can achieve. The paper is honest about this limitation but does not resolve it.
major comments (3)
- [3.6] The paper's central differentiating claim is that 'semantic persistence' (pre-declaring semantic roles for workflow artifacts before execution) is meaningfully different from 'execution persistence' (checkpoints, traces, logs captured during or after execution). Section 3.6 states: 'the model assigns declared semantic roles to workflow definitions, workflow instances, inference records, and related objects before execution begins treating them as stored data.' However, the paper does not identify any specific operation, query, or review task that requires this pre-declaration and cannot be achieved through execution persistence plus structured provenance. The three affordances listed in Section 5 (queryable history, precise review, authority clarification) are all achievable with a checkpoint system that stores typed records and a provenance store that links them. The paper's own Table 3
- [3.4] The derive/infer distinction is described as 'central' (Section 3.4) and as having operational consequences: 'A derivation may depend on an inference, but the inference should remain visible as a dependency rather than being absorbed into the derived value.' However, the paper does not specify what breaks operationally if an inference is absorbed into a derived value without a visible dependency link. What review task becomes impossible? What query fails? What correctness property is violated? Without at least one concrete scenario where the distinction has operational consequences that cannot be replicated by simply tagging model outputs in a provenance store, the derive/infer boundary risks being a naming convention rather than a semantic boundary with enforceable consequences. The paper should either (a) identify a concrete task that requires the distinction, or (b) reframe the claim从
- [3.5] The scoped reuse rule in Section 3.5 — where the executor queries the knowledge substrate for a compatible prior panel-record and reuses its choice instead of re-presenting the deliberation — is presented as a feature of semantic persistence. But this is functionally equivalent to checkpoint-based resumption with compatibility checks, which LangGraph already supports (Section 2.6 describes LangGraph's 'time travel' and checkpoint-based persistence). The paper should clarify what the semantic object model adds beyond a typed interface over existing checkpoint mechanisms. If the answer is 'nothing operationally, but the model provides a cleaner conceptual framework,' that is a legitimate but weaker claim than the paper currently implies.
minor comments (5)
- [Section 4, Appendix D] The exploratory vocabulary scan of 77 artifacts is described as a 'vocabulary-design probe' and the authors are careful to state it is not empirical validation. However, the scan's methodology is underspecified: the scoring criteria (0-3 checklist) are not reproduced, and the selection criteria for the corpus are vague ('selected, qualitative, and heterogeneous'). A brief description of the scoring rubric would help readers assess whether the vocabulary refinements (e.g., separating approval from panel) are well-grounded.
- [Table 2] The 'capability / action' and 'handoff / promotion' primitives are marked as 'Candidate refinement' in the Associated object or relation column. This is confusing because it is unclear whether these are part of the proposed model or not. If they are candidates for future inclusion, they should be separated from the core vocabulary table or clearly annotated as provisional.
- [Section 3.4] The statement that 'Non-LLM nondeterminism, such as randomized algorithms or stochastic simulations, is not infer; it can be treated as derive when the seed, inputs, algorithm version, and replay policy are explicit' is reasonable but raises a boundary question: what about external API calls that return different results over time (e.g., a web search)? These are neither deterministic nor LLM-mediated judgment. The paper does not address this common case, and the derive/infer binary framing may be insufficient for it.
- [Appendix B] The pseudo-Lisp workflow definition is helpful as an illustration but would benefit from a brief comparison with how the same workflow would be expressed in an existing system (e.g., LangGraph). This would make the contribution more concrete and help readers understand what the semantic object model adds expressively.
- [Section 2.7] The term 'agent harness' is treated as practitioner framing rather than a settled technical concept. The paper cites O'Reilly and HumanLayer blog posts. While this is transparent, the term is used in the paper's own architecture description (Section 1). The paper should either define it more precisely or avoid relying on it as a technical term.
Circularity Check
No circularity detected: conceptual model with no fitted predictions, parameter-fitting claims, or self-citation chains.
full rationale
This paper is a conceptual model proposal that does not fit the circularity framework. It makes no fitted predictions, no parameter-fitting claims, and no derivations that reduce to inputs by construction. The model is defined axiomatically through its vocabulary (derive, infer, workflow-definition, workflow-instance, etc.) and its four claims are design commitments, not empirical or mathematical results. The paper explicitly states it is 'a conceptual model proposal, not an empirical study or a formal calculus' (Section 1). Self-citation is minimal: the authors cite external systems (LangGraph, AgentSPEX, DSPy, W3C PROV, Flows/aiFlows, WorkflowLLM) as prior art and do not invoke their own prior work as load-bearing justification. The Lisp-inspired notation is explicitly framed as an 'explanatory lens, not implementation commitment.' The derive/infer distinction is a definitional choice, not a derived result that could be circular. The PROV-DM mapping in Appendix C is comparative, not a claim of derivation from PROV. No step in the paper's argument chain reduces to its inputs by construction, and no self-citation chain is load-bearing. The derivation is self-contained against external benchmarks and prior work. Score: 0.
Assumptions & free parameters
free parameters (1)
- None (conceptual model)
assumptions (4)
- ad hoc to paper Workflow definitions, instances, and inference records can and should be represented as persistent typed knowledge objects in a shared substrate.
- ad hoc to paper The derive/infer distinction is a meaningful and enforceable semantic boundary.
- ad hoc to paper Pre-declaring semantic roles for workflow artifacts before execution provides benefits over post-hoc provenance capture.
- domain assumption Local structural flow (continue, repeat, stop, accept, reject, defer) is sufficient for the target class of workflows.
invented entities (3)
-
Knowledge substrate
-
Panel-record
-
Context-snapshot
Cite this review
Pith. "Pith review of Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows." pith.science (2026). https://pith.science/paper/76RVDHLW
@misc{pith2026260708740,
author = {Pith},
title = {Pith review of: Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows},
year = {2026},
howpublished = {\url{https://pith.science/paper/76RVDHLW}},
note = {Machine review of arXiv:2607.08740}
}
read the original abstract
Large language model (LLM) applications increasingly use explicit workflows for tool use, retrieval, branching, checkpointing, and human approval. Existing workflow systems already address many execution concerns. This paper proposes a Lisp-inspired but language-independent conceptual model: symbolic forms, object identity, and live-image thinking are used as explanatory lenses, not implementation commitments. In this model, workflow definitions, workflow instances, inference records, context snapshots, and dependency relations are represented as persistent knowledge objects in a shared knowledge substrate. Its central semantic distinction is between derive and infer: derive is deterministic computation over available state; infer is mediated LLM judgment under declared context and executor-controlled capability policy. The result is a preliminary conceptual account of semantic persistence: workflows do not merely produce knowledge and leave traces, but can themselves be represented as inspectable, resumable, and reviewable knowledge objects, while formal transition semantics remain future work.
Figures
Reference graph
Works this paper leans on
-
[1]
A Comparative Study of DSL Code Generation: Fine-Tuning vs. Optimized Retrieval Augmentation
https://arxiv.org/abs/2407.02742. AgentSPEX: An Agent SPecification and EXecution Language
-
[2]
A Provenance Model for Control-Flow Driven Scientific Workflows
“A Provenance Model for Control-Flow Driven Scientific Workflows. ”Data & Knowledge Engineering 131–132: 101877. https://doi.org/10.1016/j.data k.2021.101877. Cabot, Jordi
-
[3]
Jupyter’s Archive: Searchable Output Histories for Computational Notebooks. UCB/EECS-2019-72. EECS Department, University of California, Berkeley. https: //www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-72.html. Davidson, Susan B., and Juliana Freire
work page 2019
-
[4]
Provenance and Scientific Workflows: Challenges and Opportunities
“Provenance and Scientific Workflows: Challenges and Opportunities. ” Proceedings of the 2008 ACM SIGMOD International Conference on Man- agement of Data . https://doi.org/10.1145/1376616.1376772. Ding, X. et al
-
[5]
https://doi.org/10.1145/3652620. 3687782. 37 Workflow as Knowledge: Semantic Persistence for LLM-Mediated Workflows Gabriel, Richard P., Jon L. White, and Daniel G. Bobrow
-
[6]
The Dexter Hypertext Reference Model
“The Dexter Hypertext Reference Model. ” Communi- cations of the ACM 37 (2): 30–39. https://doi.org/10.1145/175235.175237. HumanLayer
-
[7]
Academic Research Skills for Claude Code . V. v3.13.0. Released. https: //doi.org/10.5281/zenodo.20696614. Josifoski, Martin, Lars Klein, Maxime Peyrard, et al
-
[8]
Flows: Building Blocks of Reasoning and Collaborating AI
Flows: Building Blocks of Reasoning and Collaborating AI . https://arxiv.org/abs/2308.01285. Khattab, Omar et al
Show all 21 references
-
[9]
https://arxiv.org/abs/2310.03714
DSPy: Compiling Declarative Language Model Calls into Self- Improving Pipelines. https://arxiv.org/abs/2310.03714. Kiczales, Gregor, Jim des Rivieres, and Daniel G. Bobrow
-
[10]
MIT Press
The Art of the Metaobject Protocol. MIT Press. https://mitpress.mit.edu/9780262610742/the-art-of-the-metaobject-protocol/ . LangGraph
-
[11]
https: //arxiv.org/abs/2505.13453
Pel: A Programming Language for Orchestrating AI Agents . https: //arxiv.org/abs/2505.13453. Mosqueira-Rey, Eduardo, Elena Hernández-Pereira, David Alonso-Ríos, José Bobes-Bascarán, and Ángel Fernández-Leal
-
[12]
Human-in-the-Loop Machine Learning: A State of the Art
“Human-in-the-Loop Machine Learning: A State of the Art. ” Artificial Intelligence Review 56: 3005–54. https://doi.org/10.1007/s10462-022-10246-w . Nelson, Theodor H
-
[13]
Complex Information Processing: A File Structure for the Complex, the Changing and the Indeterminate
“Complex Information Processing: A File Structure for the Complex, the Changing and the Indeterminate. ” Proceedings of the ACM 20th National Conference , 84–100. https://doi.org/10.1145/800197.806036. O’Reilly. 2026a. Agent Harness Engineering . O’Reilly Radar. https://www.or...
- [14]
-
[15]
Samuel, Sheeba, and Birgitta König-Ries
https://arxiv.org/abs/2604.25917. Samuel, Sheeba, and Birgitta König-Ries
-
[16]
https://doi.org/10.1109/ACCESS.2018.2887
2018 doi
-
[17]
LLM Agents for Interactive Workflow Provenance: Reference Archi- tecture and Evaluation Methodology
Souza, Renan et al. 2025a. “LLM Agents for Interactive Workflow Provenance: Reference Archi- tecture and Evaluation Methodology. ” WORKS at ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis . https://doi.org/10.1145/37 31599.376...
-
[18]
https://arxiv.org/abs/2506.10021
From Tool Calling to Symbolic Thinking: LLMs in a Persistent Lisp Metaprogramming Loop. https://arxiv.org/abs/2506.10021. W3C
-
[19]
W3C Recommendation
PROV-DM: The PROV Data Model . W3C Recommendation. https://www.w3.org/T R/2013/REC-prov-dm-20130430/ . Walker, Janet H., David A. Moon, Daniel L. Weinreb, and Mike McMahon
2013
-
[20]
Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al
https://arxiv.org/abs/2411.05451. Yao, Shunyu, Jeffrey Zhao, Dian Yu, et al
-
[21]
https://arxiv.org/abs/2210.03629
ReAct: Synergizing Reasoning and Acting in Language Models. https://arxiv.org/abs/2210.03629. 39
Reviewed July 10, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.