{"id":"8882b7ce-8b59-4c6a-9d70-ef87817e51ec","arxiv_id":"2608.05412","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Introduces four semantic-isolation anomalies and a partial order of isolation levels for durable AI workflows, backed by a SemIso middleware prototype and a public-codebase audit.","lead":"This paper defines four ways a long-running AI workflow can become internally inconsistent when prompts, models, indexes, or tools change while the workflow is paused or running, and proposes middleware that detects or blocks these inconsistencies. It also reports that 7.4 percent of durable workflows in popular public LangGraph projects show signs that could trigger such inconsistencies.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SSI's compatibility guarantee presupposes complete, truthful, current contracts; the motivating case of 'same tool name, new meaning' has no contract to check, so the central guarantee is conditional on an external declaration oracle.","rationale":"The paper's strongest claim is that semantic isolation is necessary and implementable, with SSI preventing all four anomalies at negligible latency. The reader identified the weakest assumption as reliance on declared, accurate contracts and recommended a more prominent limitation statement; my stress-test concurs and sharpens the point. Section 4 is explicit that SemIso enforces but does not infer contracts, so there is no internal inconsistency in the formalism. However, the motivating scenario depends on detecting semantic changes that are not necessarily declared: a tool name's meaning can change, an embedding model can drift while preserving dimensions, and policies can be updated without a registered compatibility predicate. In all such cases, SemIso's compatibility check evaluates a predicate that does not exist or is stale, reducing SSI to sticky binding with unchecked composition. The evaluation does not test this failure mode; it declares the relevant contract for the equal-dimension embedding trick, so the microsecond latencies do not establish protection against undeclared incompatibility. This does not invalidate the paper's useful formalization—it correctly defines compatibility skew relative to declared conditions—but it does mean the central 'semantic isolation' claim should be scoped as 'isolation over declared semantic contracts,' with the declaration-oracle assumption listed as a first-class limitation rather than a parenthetical cost. On the positive side, the independence of the three guarantees, the anomaly reproductions in a real framework, and the direct comparison against static pinning are well-executed and support the paper's conceptual contribution. The contract-declaration concern is a scope limitation, not a fundamental flaw, so the conditional verdict stands without change.","tokens_in":8821,"tokens_out":5038,"duration_ms":49587,"concrete_test":"Build the Figure-1 retrieval workflow under SemIso with SSI using Idx1 built on Emb1 and Emb2 of identical dimensions but a different embedding space, and do not declare any Idx1-requires-Emb1 contract. Run the workflow; if SemIso admits Emb2 against Idx1 and returns the wrong controlled document, the compatibility guarantee fails for undeclared incompatibility. Repeat the test with a stale contract: declare Idx1 requires Emb1, deploy a new index version that semantically requires Emb2 but leave the old contract unchanged, and check whether SemIso blocks the correct resolution or admits the wrong one. If either run succeeds, the C1 guarantee is not 'semantic' in the sense the motivating example requires.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central guarantee C1 (and therefore SSI's claim to prevent all four anomalies) collapses to C0 whenever a semantic incompatibility is not captured by a declared contract. Section 4 states: 'Owners or applications declare contracts such as index I1 requires embedding model E1; SemIso enforces but does not infer them.' Section 3 concedes 'Compatibility needs trustworthy contracts,' but this is treated as a cost, not as a fundamental limit on what 'semantic isolation' can mean. The motivating example is precisely a change of meaning behind a stable name ('change what a risk score means behind the same tool name') with no machine-readable contract covering that meaning. The evaluation evades this by declaring the relevant contract: the equal-dimension embeddings used in Section 5.2 are caught only because Idx1's dependency on Emb1 is registered. If a provider updates a model, index, prompt, or tool without registering the new contract, or if two resources are semantically incompatible but no predicate expresses it, SemIso under SSI will admit the incompatible combination, and read skew, context escape, and merge skew can also pass when the underlying incompatibility is undeclared. The paper is internally consistent—compatibility skew is defined as a declared predicate evaluating false—but the headline claim of 'semantic isolation' is broader than what is actually enforced: it is isolation over declared contracts only. This is the load-bearing soft spot because every real-world benefit claimed in the abstract and the audit depends on the completeness and truthfulness of declarations that the system itself cannot verify.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper formalizes 'semantic isolation' for durable AI workflows. It defines four trace-level anomalies—semantic read skew, compatibility skew, context escape, and merge skew—each with an automatically checkable condition and a witness history, and it argues that no anomaly implies another. It then encodes an isolation contract as a triple ⟨S,C,I⟩ of resource stability, cross-resource compatibility, and continuation inheritance, deriving a four-level partial order: Semantic Read Committed, Repeatable Semantic Read, Compatible Read, and Semantic Snapshot Isolation (SSI). The prototype, SemIso, enforces these levels in LangGraph middleware via a checkpointed semantic manifest, a compatibility resolver, continuation inheritance, and merge admission, with an opaque-resource abort/capture/taint policy. The evaluation has three parts: a conservative manual audit of the 100 most-starred executable LangGraph codebases (4 exposed codebases overall; 7.4% of the 54 durable ones), a controlled LangGraph+Qdrant reproduction showing that the default runtime admits all four anomalies while SSI prevents them (and static pinning fails in an open-world discovery control), and microbenchmarks reporting p95 latencies of 4.6 μs for resolution and 88.2 μs for merge validation at 128 identities and 16 branches. The paper honestly scopes the read-only case and defers stronger write/effect guarantees to future AI-workflow serializability.","tokens_in":9097,"tokens_out":18485,"duration_ms":158623,"significance":"The contribution is timely and useful if the claims hold: it gives agents-and-workflow practitioners a shared vocabulary and a runtime contract for a real failure mode—semantic drift behind stable resource names—using the methodology that made ANSI SQL isolation levels productive. The strengths are concrete: the anomaly conditions are mechanically checkable; the witness histories ground the independence claims; the system evaluation is carefully controlled (deterministic adapters, a fixed deployment schedule, and equal-dimension embeddings so that the compatibility failure is semantic rather than a type error); the audit is conservative and explicitly labeled as evidence of enabling conditions, not observed anomalies; and the latency figures transparently cover metadata processing only, with the operational cost of retaining historical versions acknowledged. The main limitation, which bounds the significance, is that compatibility enforcement operates over declared contracts and previously bound identities; the paper demonstrates isolation over that scope rather than over arbitrary semantic drift.","major_comments":[{"comment":"The headline claim exceeds the enforced guarantee. The motivating example in §1 turns on 'change what a risk score means behind the same tool name,' but this failure mode has no formal counterpart in §2 unless a machine-readable contract predicate captures risk-score semantics. Because C1 evaluates only declared contracts ('Owners or applications declare contracts such as index I1 requires embedding model E1; SemIso enforces but does not infer them,' §4), a late-discovered tool whose meaning has drifted is resolved and bound without any check, and SSI admits the combination of saved evidence with the new meaning. The §5.2 evaluation never exercises this path: the compatibility-skew scenario is caught only because the equal-dimension embedding dependency of Idx1 on Emb1 is registered, so the experiments confirm enforcement inside the declared-contract regime. The paper is internally consistent—compatibility skew is defined as a declared predicate evaluating false—but the abstract's promise ('stable names can acquire new behavior') is broader than what is enforced. Please (a) state at the abstract and introduction level that the guarantees cover declared contracts and previously bound identities rather than arbitrary semantic drift; (b) add a fail-closed admission policy for resources whose relevant semantic properties are not covered by any declared contract, with its costs; and (c) add a §5.2 run with an undeclared semantic change to show the failure-open outcome and the effect of the policy.","section":"§1, §3, §4, §5.2"},{"comment":"The interaction between I1's merge admission and the isolation profiles is under-specified. Section 4 states that, before releasing merged output, the manager 'rejects conflicting bindings, violated contracts, unavailable dependencies, or disallowed taint,' which reads as unconditional, yet Table 2 lists CR ⟨S0,C1,I1⟩ as allowing merge skew. The statements are reconcilable only if 'conflicting' is profile-relative: under S0, two versions of one identity in the union are admissible, so the union is not 'conflicting,' while under S1 the same union is rejected. Please state explicitly, for each profile, which admission checks apply and what 'conflicting bindings' means under S0 versus S1, so that Table 2 and Section 4 cannot be read as inconsistent.","section":"§4 (Continuation manager) and Table 2"}],"minor_comments":[{"comment":"Since SSI is defined in §3 as the profile combining S1, C1, and I1, and those guarantees by construction exclude the four anomalies, the SemIso(SSI) row in Table 3 demonstrates implementation conformance to the specification rather than an empirically discovered property. This is legitimate methodology, but §5.2 should frame the row as a conformance check so the result is not read as a predictive finding.","section":"§5.2, Table 3"},{"comment":"The independence of S, C, and I is argued with one-line scenarios ('A permanently bound but incompatible embedding/index pair separates S1 from C1'), but the partial-order claim is a stated contribution; formal witness histories for each pair of guarantees, in the style of the §2 anomaly witnesses, would make the derivation rigorous.","section":"§3"},{"comment":"The merge-skew detection condition uses 'admissible common extension' without a formal definition; the paper should define it explicitly (for example, an extension of the union that is dependency-closed and satisfies all declared compatibility predicates) so the condition is mechanically checkable as promised.","section":"§2 (merge skew)"},{"comment":"The anomaly predicates are defined over execution traces, while SemIso enforces them through checkpointed manifests; a short invariant or lemma stating that the manifest-based checks (sticky bindings, dependency closure, inheritance propagation, merge admission) preclude the corresponding trace predicates would tighten the link between the formal model and the implementation.","section":"§2 and §4"},{"comment":"There is a typographical glitch early in §1: 'One report, incompatible assumptions— —' contains a doubled em-dash, and the opening example would benefit from matching the four panels of Figure 1 more explicitly to the notation in §2.","section":"§1 (typography)"}],"recommendation":"major_revision","confidential_remarks":"This is a single-author arXiv preprint (August 2026) with an AI-use disclaimer; the technical content, not the authorship or polish, is the basis of my assessment. The fit with the journal's scope is good: the paper is squarely in data management for AI systems. The novelty is moderate—classic database isolation methodology applied to durable AI workflow resources—but the formalization is clean, the anomaly conditions are checkable, and the systems evaluation is carefully controlled; I consider this adequate for the venue. The reference list draws on relevant 2026 arXiv preprints and standard database literature, and I found no obvious omission of prior art in versioned ML, durable workflows, or agent transactions. My recommendation of major_revision rests on the claim-scope issue in major comment 1 and the profile-relative merge admission in major comment 2; both are fixable without changing the formal core."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nRead this paper on durable AI workflows and semantic isolation. The core idea is right: checkpoints preserve state but not the semantic assumptions that give state meaning, and that is an isolation problem. The paper defines four anomalies—semantic read skew, compatibility skew, context escape, merge skew—with trace definitions and witness histories showing they are independent. That is genuinely new and useful. The S/C/I lattice is a clean composition, methodologically in the spirit of ANSI SQL isolation levels, and the named levels are clearly specified. SemIso is real middleware that enforces the profile it claims, with p95 checks in tens of microseconds. The evaluation is carefully controlled with deterministic adapters and honest about what it measures.\n\nThe soft spots are real but not fatal. The biggest is that the entire compatibility guarantee rests on declared contracts. Section 4 says owners declare contracts and SemIso does not infer them. The paper's motivating example—'change what a risk score means behind the same tool name'—has no machine-readable contract covering that meaning. So SSI, as implemented, is isolation over declared contracts, not over all semantic incompatibilities. The stress-test note is right about that. The paper mentions this at Section 3 as a cost, but understates the consequence: the headline claim is broader than the enforced guarantee. A prominent limitation statement would fix most of it.\n\nThe empirical audit is honest: 7.4% of 54 durable codebases is a small sample, and the paper correctly frames these as enabling conditions, not actual anomalies. The robustness check on 173 repos is a nice touch, but confidence intervals are wide and the number is easy to overquote.\n\nThe isolation levels are definitions, so SSI preventing all four anomalies is true by construction rather than empirical discovery. That is fine—the ANSI isolation literature works the same way—but worth remembering when citing.\n\nWho is this for? Researchers building durable agent frameworks and transaction abstractions for AI workflows. They will get value. The paper deserves a serious referee. My recommendation: send to review, but the author should address the contract-declaration caveat head-on and ideally release the prototype and audit artifacts. Conditional accept, not reject.","headline":"New formalization of semantic isolation for durable AI workflows; a real contribution, but the central guarantee depends on declared contracts that the motivating example itself lacks.","tokens_in":9601,"tokens_out":1669,"would_cite":true,"duration_ms":15038,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Long-running AI workflows can silently combine saved state with changed semantic assumptions, producing internally inconsistent results even when every call succeeds, and this paper defines and enforces an isolation contract to prevent it.","keywords":["semantic isolation","durable AI workflows","isolation levels","semantic manifest","workflow consistency","AI transactions","resource versioning","checkpointing"],"falsifier":"Under Semantic Snapshot Isolation, deploy a new version of an embedding model whose contract identifier is unchanged but whose vectors are incompatible with the bound index, then let a paused workflow query that index with the newly bound model; if SemIso admits the query, the compatibility claim is false.","tokens_in":8633,"feed_emoji":"🔄","tokens_out":5805,"duration_ms":47954,"temperature":0.7,"pith_summary":"The paper argues that durable AI workflows—executions that pause, resume, branch, delegate, and discover tools—can outlive the semantic environment in which they began. Because prompts, model aliases, indexes, policies, and tools are deployed independently behind stable names, a workflow can restore saved state and then resolve those names to different versions, producing an internally inconsistent result even though every call succeeds. The paper frames this as an isolation problem, defines four automatically checkable anomalies, and derives a partial order of isolation levels over three independent guarantees: resource stability, cross-resource compatibility, and continuation inheritance. It then presents SemIso, middleware that checkpoints a semantic manifest, propagates it through resumes, forks, and children, and validates late-discovered resources and branch merges with microsecond-scale checks. A conservative source audit of popular executable durable workflows finds 7.4 percent of durable codebases resolve live or dynamically selected semantic resources without an immutable binding, evidence that the failure mode is real.","feed_headline":"Checkpointed AI workflows can silently mix old and new semantics","feed_subtitle":"A new isolation contract blocks stale saved state from combining with changed models, indexes, and policies.","key_machinery":"The load-bearing object is the semantic manifest: a per-workflow identity-to-version mapping together with inherited compatibility obligations, stored in the checkpoint and extended online as late-discovered resources arrive. The runtime records bindings in a shared resource-version graph, resolves each requested identity to an immutable version that satisfies declared contracts, propagates the manifest to retries, resumes, children, and branches, and performs merge admission before releasing combined output. Stability, compatibility, and inheritance are independent toggles, so the isolation levels form a partial-order lattice rather than a ladder.","core_discovery":"On the paper's own terms: an AI workflow has a semantic environment that determines what saved state means, and current checkpointing preserves only control state, not that environment. The central discovery is that deployment of prompts, models, indexes, policies, and tools acts as a concurrent semantic write with respect to every active workflow, so a durable execution can combine state from one semantic cut with resources from another. The paper identifies four concrete, decidable failure modes—semantic read skew, compatibility skew, context escape, and merge skew—and shows that no anomaly implies another, so preventing them requires combining three independent guarantees: sticky version binding, dependency-closed compatibility, and inherited continuation context. The named isolation levels range from Semantic Read Committed, which permits all four anomalies, to Semantic Snapshot Isolation, which preserves one stable, compatible, inherited semantic cut and blocks all four. The paper reports that this enforcement is practical: its prototype adds p95 checks of a few microseconds and negligible end-to-end overhead on a laptop.","pith_inferences":["Beyond the paper's audit, the 7.4 percent exposure estimate is probably a lower bound, since source-visible immutable bindings are hard to verify and the same risk can be introduced through deployment configuration rather than code.","A natural extension is to learn contracts automatically from version histories or embedding similarity rather than relying on declarations, which would close the weakest assumption.","If these guarantees generalize, the same manifest-based scheme could be dropped into other durable-execution frameworks, making semantic isolation a framework-level contract rather than an application convention.","The sketched AI-workflow serializability property, which also coordinates writes and visible tool effects, would subsume semantic isolation for workflows that modify shared artifacts."],"forward_implications":["Workflow frameworks can offer application-chosen isolation levels, so a compliance report can require Semantic Snapshot Isolation while a chat agent uses Semantic Read Committed.","Checkpoint/resume systems can treat a deployment as a concurrent write and reject or recompute a merge whose branch manifests have no admissible union, instead of emitting an internally inconsistent artifact.","Dynamic tool discovery becomes safe: when a model selects a tool mid-execution, the runtime binds it, checks it against inherited constraints, and serves only the compatible retained implementation.","The four anomaly predicates give a replayable oracle for testing durable workflows against deployment schedules, independent of generated outputs.","Because static pinning only covers closed dependency universes, open-world agents need online manifest extension to keep late-discovered resources consistent."],"supporting_citations":[{"why":"Supplies the database isolation-level framework that semantic isolation extends to AI workflows.","marker":"[3]"},{"why":"Shows that the workflow framework's flow_version only pins graph logic, not external semantic resources.","marker":"[11]"},{"why":"Documents the durable, stateful AI-agent orchestration framework where the gap is demonstrated.","marker":"[12]"},{"why":"Establishes that checkpointing saves thread state but does not bind independently managed prompts, indexes, or tools.","marker":"[13]"},{"why":"Provides static pipeline lockfiles as the closed-world pinning baseline that semantic isolation extends.","marker":"[8]"},{"why":"Offers pipeline snapshots, representing closed-world static pinning that cannot cover late-discovered resources.","marker":"[24]"},{"why":"Records addressable model versions in a registry but does not prevent an inconsistent commit.","marker":"[16]"},{"why":"Documents a real embedding-model–index migration contract, motivating cross-resource compatibility constraints.","marker":"[19]"}],"fun_headline_variants":["Semantic isolation for AI: stop old state meeting new tools","AI workflows: new contract blocks stale semantic state","Four skew anomalies threaten durable AI runs—now fixed","SemIso: microsecond checks for semantic consistency","Durable AI gets isolation levels from read committed to snapshot"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire compatibility guarantee rests on owners and applications declaring complete, accurate contracts for every resource version, so a deployment that changes behavior without updating its contract, or a conflict no declared predicate captures, would pass undetected.","fun_headline_variants_meta":{"raw":{"variants":["Semantic isolation for AI: stop old state meeting new tools","AI workflows: new contract blocks stale semantic state","Four skew anomalies threaten durable AI runs—now fixed","SemIso: microsecond checks for semantic consistency","Durable AI gets isolation levels from read committed to snapshot"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000249,"raw_usage":{"total_tokens":1578,"prompt_tokens":1000,"completion_tokens":578,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":500}},"tokens_in":616,"tokens_out":578,"duration_ms":5646,"temperature":1.0,"reasoning_tokens":500,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:37:41.420913+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Under Semantic Snapshot Isolation, deploy a new version of an embedding model whose contract identifier is unchanged but whose vectors are incompatible with the bound index, then let a paused workflow query that index with the newly bound model; if SemIso admits the query, the compatibility claim is false.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the database isolation-level framework that semantic isolation extends to AI workflows."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that the workflow framework's flow_version only pins graph logic, not external semantic resources."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the durable, stateful AI-agent orchestration framework where the gap is demonstrated."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that checkpointing saves thread state but does not bind independently managed prompts, indexes, or tools."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides static pipeline lockfiles as the closed-world pinning baseline that semantic isolation extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Offers pipeline snapshots, representing closed-world static pinning that cannot cover late-discovered resources."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Records addressable model versions in a registry but does not prevent an inconsistent commit."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents a real embedding-model–index migration contract, motivating cross-resource compatibility constraints."}],"review_version":1}